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Abstract 

Most computational models of neurons assume that their electrical characteristics 
are of paramount importance. Because the speed of electrical transmission and the 
binary nature of action potentials is so similar to modern computer technology, other 
aspects of the neural substrate have been regarded as irrelevant. However, electrical 
properties of neurons are modulated in important ways by chemical mechanisms. In 
addition to mediating all long-term changes, chemical mechanisms can have short- 
term effects that change the cell's response properties. They also play important roles 
in the creation of the brain's anatomical structure during development. 

The topic of this dissertation is the interaction between electrical and chemical 
mechanisms in neural learning and development. Plasticity is prevalent at all levels in 
the brain, and it plays a central role in the establishment of the brain's structure and 
function. My central thesis is that chemical mechanisms must be taken into account 
if we are to build accurate models of neural computation. 

To demonstrate their role, two neural systems are modelled in some detail. The 
hrst is the gill withdrawal reflex circuit in Aplysia, a marine snail. The mechanisms 
underlying habituation, sensitization, and associative learning in this circuit have 
been extensively studied. These mechanisms display an intricate interaction between 
electrical conductances and chemical processes inside the synaptic terminals. 

The second system modelled is the early visual pathway. The initial connections 
formed between structures in this pathway are highly disordered. A detailed map 
topography is produced using a combination of chemical and electrical mechanisms 
to sort the axonal projections. A simplified model of individual neurons is used in 
this simulation so that network interactions can be studied. However, some care is 
taken to remain compatible with experimental results. 

Thesis Supervisor: Rodney A. Brooks 
Title: Associate Professor, EECS 
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The Genealogy of Brigid 

Brigid of the mantles, 

Brigid of the peat heap, 

Brigid of the twining hair, 

Brigid of the augury, 

Brigid of the white feet, 

Brigid of calmness, 

Brigid of the kine, 

Brigid, daughter of the Dagdah: 

I am under the shielding 

of Good Brigid each day, 

I am under the shielding 

of Good Brigid each night. 

Brigid is my companion, 

Brigid is my maker of song, 

Brigid is my helping Goddess, 

my sustenance, 

my guide. 

Every day and every night 

that I recite the genealogy of Brigid, 

I shall not be killed, 

I shall not be harried, 

I shall not be put into a cell, 

I shall not be wounded, 

I shall not be anguished, 

I shall not be ravaged, 

I shall not be blinded, 

I shall not be left bare 

or forgotten. 

No fire shall burn me, 

Nor sun shall burn me, 

Nor moon shall blanch me, 

Nor lake shall drown me, 

Nor flood shall drown me, 

Nor briny sea shall drown me, 

Nor dart of faerie sting me, 

Nor seed of faerie lift me, 

Nor earthly being destroy me. 

For I am the child of Poetry, 

Poetry, child of Reflection, 

Reflection, child of Meditation, 

Meditation, child of Lore, 



Lore, child of Research, 

Research, child of Great Knowledge, 

Great Knowledge, child of Intelligence, 

Intelligence, child of Comprehension, 

Comprehension, child of Wisdom, 

Wisdom, child of Brigid. 
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Chapter 1 
Introduction 



A study of the brain's architecture and learning mechanisms can be extremely useful 
in understanding the nature of intelligence and adaptive behavior. However, in the 
past, the usefulness of biological examples has been limited by the primitive state 
of our understanding of both computational systems and neuroscience, and by the 
limited interaction between computational modellers and experimentalists. 

Research is progressing rapidly in both Computer Science and Neuroscience, and 
both fields benefit by maintaining communication between them. Computational 
modelling in neuroscience helps to understand and integrate experimental data and 
to test hypotheses about the function of neural circuits, and it provides a framework 
to focus the direction of future research. In artificial intelligence and connectionist 
research, the articulation of underlying principles of development in neural systems 
can provide important insights and design principles for the construction of intelligent 
systems. 

This thesis describes mathematical and computational models of several known 
biological learning mechanisms. The methodology is to construct computer models 
of example systems utilizing two types of learning, classical conditioning in Aplysia, 
and long-term potentiation (LTP) in mammals. These circuits are studied in terms of 
both chemical interactions within and between neurons and overall network structure. 
Their capabilities and limitations are explored through simulation and analysis. 

This work is, of necessity, speculative, and serves primarily to sketch out an area 
of study and identify some interesting problems, and to fill in a few details that point 
the way toward the kind of answers that are likely to be found as more work is done 
in this area. 



1.1 Goals 

The goal of this research is to better understand the relationship between structure 
and function in the nervous system: that is, between the biological mechanisms un- 
derlying different types of learning and their computational properties. 

15 
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The questions I am interested in include: 

1. What is the nature ol the computational primitives in the brain, and what kinds 
of algorithms can be implemented efficiently with them? 

2. What mechanisms underlie different forms of learning and memory? 

3. What is the relative contribution of network properties and properties of indi- 
vidual neurons to the computational capabilities of the brain? 

4. In what ways can learning be incorporated into structured networks? 

5. How are biological learning mechanisms similar to or different from current 
computational learning models? 

It is my hope that the neural systems described and modelled in this work provide 
some initial insights as to what the answers those questions may be. 

1.2 Chapter Organization 

The central chapters of this document take a detailed look at two neural systems 
which illustrate a variety of learning processes in neurobiology. 

Chapter 2 discusses the process of modelling in neuroscience, at a philosophical 
level. 

Chapter 3 describes a circuit controlling the gill withdrawal reflex in Aplysia, 
a marine snail, and presents a simulation and and analysis of the mechanisms un- 
derlying observed habituation, sensitization, and associative learning in the reflex. 
These mechanisms display an intricate interaction between electrical conductances 
and chemical processes inside the synaptic terminals. 

The second simulation models the formation of ordered topographic connections 
between different structures in the early visual pathway. Chapter 4 describes a sim- 
ulated model of topographic map formation in the visual system, and Chapter 5 
presents a mathematical analysis of the system and the constraints necessary to en- 
sure its success. 

The final chapter draws the various parts of this research together and summarizes 
the insights into the particular brain systems studied. It discusses principles that are 
common to the different systems, and which may turn out to be more generally true 
of biological systems that mediate adaptive behavior. 



Chapter 2 

Modelling in Neuroscience 



My primary reason for choosing this topic was a deep curiosity about the brain and 
how it works, particularly with regard to the nature of neural computation. One of 
the most interesting aspects of my undergraduate education was the explication of 
all the layers of abstraction in computers, from the internal electrical properties of a 
silicon transistor to a high-level programming language. I started this work with the 
goal (however unrealistic) of doing the same with the brain: understanding all the 
layers of neural processing from the underlying chemical processes to its higher-level 
functioning. 

Just as the nature of the underlying hardware directs and constrains the character 
of computer languages, the nature of the neuronal substrate constrains the character 
of the computational primitives and "programming languages" used in the brain. For 
this reason, characterizing the behavior of neurons is crucial for developing an under- 
standing of neural computation. Mechanisms involved in learning and development 
are emphasized throughout this work in recognition of the prevalence of plasticity at 
all levels in the brain, and its central role in the establishment of the brain's structure 
and function. 

Learning, in the context of traditional neural networks research, is conceptualized 
as permanent synaptic changes in response to information encoded in the neuron's 
electrical activity. However, neurons and their synaptic connections change over time 
in a wide variety of ways, only a few of which fit this model. Although neural plasticity 
is often activity- dependent, many other processes in the cell, such as chemical and 
mechanical properties, also play important roles in the cell's development over time. 

Learning in the brain is thus a far richer and more complex phenomenon than is 
recognized by traditional models. A primary thesis of this work is that a recognition of 
the multiplicity of mechanisms for plasticity in the cell is essential to understanding 
the way the system functions as a whole. Because most previous models of the 
functional characteristics of neurons have concentrated exclusively on their electrical 
properties, the focus here is on chemical mechanisms. 
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2.1 The Value of Models 

The field of neuroscience has progressed very rapidly in the past few years. A range 
of new experimental techniques allows researchers to probe the internal mechanisms 
of neurons in far more detail than was possible even five or ten years ago. 

Even so, biological structures are so complex, and biochemical interactions are so 
intricate and difficult to isolate, that many aspects of the physical system must still 
be inferred from insufficient data. This is one of the major challenges of the held, 
and it limits the uses to which modelling can be put. 

Until biological systems are understood far better than they are at present, a 
biological model, however rigorous it may be mathematically, is at best a working 
hypothesis. The question of whether a particular model is "correct" is therefore of 
far less interest than that of whether it is useful. How well does it reveal the gaps in 
our knowledge, and suggest questions to direct further experimentation? 

At worst, a biological model is simply wrong. It can be difficult to keep track 
of the many many details of what is known about a neural system, and integrate 
them into a model that remains compatible with the current state of experimental 
research. The goal of biological modelling is to organize this information in a way 
that reveals some underlying structure or functional principle while maintaining a 
plausible correspondence between the mathematical properties of a simulation and 
its supposed biological substrate. 



2.2 Is it Real or is it Artifact? 

It can often be difficult to distinguish, when looking at a system in which everything 
interacts with everything else, whether a given effect is an integral part of the system's 
functional design, or an artifact that is due to undesired but unavoidable constraints 
of the physical substrate. However, there are a number of properties which can be 
examined to help distinguish between these situations. 

• One question we can ask is how many cellular resources participate in producing 
the effect, or would be necessary in order to avoid it. An effect which requires 
the coordination of many different processes is more likely to be significant than 
one which results from a single mechanism. Conversely, if a large number of 
cooperating processes would be necessary to prevent an effect from occurring, or 
if there is evidence for the presence of processes which minimize the effect, this 
would be a strong argument for classification of that effect as an unavoidable 
artifact. 

• Another question is whether the behavior results from a cellular mechanism 
whose function in other systems is well established. Protein phosphorylation as 
a response to extracellular signals, for example, was hrst studied in the control 
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of glycogen metabolism, and has been established as an important mechanism 
for modulating a wide variety of metabolic processes in many different cell types 
[50]. This mechanism is therefore a likely candidate for mediating a neuron's 
metabolic responses to neurotransmitters. 

• Given some parameter which activates and deactivates a particular process, a 
correspondence between the range of variability necessary to control the process 
and the observed range of variability of that parameter in vivo 1 may be an 
indication of functional significance. 

Two examples involving the NMDA receptor, a protein involved in interneuronal 
communication, illustrate this point. The action of NMDA receptors is strongly 
potentiated in response to very small concentrations of glycine [23]. However, 
this effect reaches saturation even with trace amounts of glycine, comparable 
to the basal levels found in the extracellular fluid, so there is no evidence for 
a differential response to a controlled variation in the glycine concentration 
in vivo. This does not, however, rule out the possible significance of glycine 
potentiation, as there may be controlling processes which have not yet been 
discovered. 

Basal levels of magnesium also affect the NMDA receptor, by blocking its ac- 
tivity in a voltage-dependent manner. The voltage at which magnesium ceases 
to act as an activity block falls well within the cell's normal operating range 
[34]. In this case, therefore, the evidence supports the hypothesized functional 
significance of the magnesium block. 

• Another indication that may be useful in determining whether a mechanism is 
functionally important is its magnitude relative to other mechanisms that affect 
the same behavior. An example would be a comparison of synaptic depression 
to habituation. In synaptic depression, neurotransmitter release is reduced by 
only ten or fifteen percent following prolonged firing. In habituation in Aplysia, 
by contrast, synaptic transmission is reduced to thirty percent of its initial value 
by only a few training trials [30]. 

The absence of a large effect, however, does not necessarily rule out the func- 
tional significance of a candidate cellular mechanism. The lack of positive ev- 
idence could be due to limitations in the experiment. For example, localized 
effects may be lost in the noise if the measuring technique is not specific enough. 

• A highly specific response to a particular chemical, and a lack of response to 
other similar chemicals, may also indicate functional significance. 

Although none of these arguments is conclusive, and in many cases new experi- 
ments may provide evidence which counters our current intuitions, they nonetheless 



^^That is, in the living animal, as opposed to the petri dish. 
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can improve our ability to distinguish important effects from artifacts. The mecha- 
nisms discussed in this dissertation all satisfy at least one of these indications for a 
functionally significant effect. 

2.3 Selecting a System for Modelling 

There are a number of properties of neural systems which, if known, make it easier 
to model learning phenomena in that system. Some of these are: 

1. A description of the input and output behavior of the circuit, both before and 
after learning. 

2. A description of the internal and external connections in the circuit. 

3. Information about the behavior of the synapses, for example, the strength of 
the connections, whether they are excitatory or inhibitory, and what forms of 
learning are present. 

4. A high-level description of the function the system performs. 

5. Evidence for the presence and importance of learning phenomena in the system. 

Both the Aplysia gill withdrawal reflex and map formation in the early visual 
system have been extensively studied and are relatively well-understood. Because 
many of the mechanisms involved are widespread in other brain regions and across 
species, it is expected that the principles discovered here will generalize to other 
learning situations as well. 



Part I 



Classical Conditioning in Aplysia 
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Chapter 3 

Synaptic Plasticity in Aplysia 



This chapter presents a simulation and analysis of mechanisms underlying short-term 
learning in the marine snail Aplysia. 

One purpose of this work is to test the conceptual models described by neurobiol- 
ogists to discover whether the hypothesized mechanisms are adequate to explain the 
observed learning phenomena. The simulation demonstrates that the mechanisms de- 
scribed in the literature can produce some simple aspects of the experimental results, 
but there are many details which it cannot explain. For example, inconsistencies in 
learning and recovery rates when different training protocols are used require a more 
complex explanation. 

Two specific mechanisms are analyzed in detail to explore the relationship between 
the macroscopic behavior of the system and the underlying mechanisms. These are 
the process governing the relationship between Ca ++ influx and transmitter release 
and the chemical cascade which controls the effect of the inter-stimulus interval (ISI) 
on the strength of associative learning. The analysis predicts certain parameters of 
those processes, such as chemical reaction rates, which can be tested experimentally. 

Studies of learning in Aplysia provide evidence for a specific biological implemen- 
tation of classical conditioning phenomena, which makes use of both chemical and 
electrical properties of the synapses. Similar processes may be used to control a broad 
range of neural responses. 

3.1 The Nature of the Beast 

Aplysia is a marine gastropod mollusc, often (inaccurately) called a "sea slug". Eric 
Kandel, one of the more prominent experimentalists working with Aplysia, describes 
the creature as follows: 

As is readily appreciated from its proud gait, illustrated in Figure 3-1, 
Aplysia californica is a large, attractive, and obviously intelligent beast. 
It is just the sort of animal one would immediately select for studies of 
learning. [28, page 6] 

23 
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Figure 3-1: Aplysia's "proud gait" (from [28, page 7].) 
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Aplysia is currently in demand for studies in neuroscience for two major reasons. 

The first is that its nervous system has a number of properties that make it easy to 
study [27]. Its neurons are relatively large, and the organization of its nervous system 
is consistent enough that individual neurons can be reliably identified from one animal 
to the next. Synapses are electrically close to the cell body, so intracellular recording 
of synaptic potentials is possible. This means that the interneuronal connections 
— the "wiring diagram" — of the animal can be determined through intracellular 
recording at the cell body. 

The second reason is that it learns. 

3.1.1 What Does Aplysia Learn? 

The learning phenomena observed and studied in Aplysia and other gastropod mol- 
luscs fall under the category of classical conditioning responses. Classical conditioning 
builds on innate reflex responses, which are modified in response to experience. A 
reflex consists of a sensory stimulus for which the animal exhibits a characteristic 
behavioral response. 

The Phenomena of Classical Conditioning 

First-order classical conditioning responses 1 include habituation, dishabituation, sen- 
sitization, associative learning, extinction and recovery [9]. In habituation, repeated 
application of a stimulus results in attenuation of the reflex response. This atten- 
uation can last for an extended period. Dishabituation and sensitization are very 
similar, as both involve an enhancement of the reflex response following a strong 
stimulus. This stimulus need not be the same as the one that normally elicits the 
reflex response. The difference between the two is that dishabituation acts on an 
already-habituated response, whereas sensitization can act regardless of whether the 
reflex is habituated. 

Associative learning is the best-known phenomenon of classical conditioning. 
There are two stimuli involved in associative learning, called the unconditioned stim- 
ulus (US) and the conditioned stimulus (CS). The US naturally evokes the reflex 
response (UR). If the US and the CS are paired in a sequence of learning trials, the 
CS alone will begin to evoke the reflex response (CR). 

In extinction, a CR gradually disappears following repeated application of the CS 
unpaired with the US. After a CR has been extinguished, it often reappears after a 
period of rest. This response is called recovery. 

Higher-order classical conditioning phenomena include second-order conditioning, 
where a new stimulus is conditioned using an already conditioned stimulus-response 
pair in place of a primary reflex response, generalization to novel stimuli that are 



^^That is, responses which involve only a single conditioned stimulus. 
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similar to the CS, and blocking, where an already conditioned stimulus prevents con- 
ditioning of a new stimulus. 

The Gill Withdrawal Reflex 

Classical conditioning in Aplysia is studied using the gill withdrawal reflex. One of 
the more prominent features of the molluscan body is the mantle, located on its back 
(see Figure 3-2). The mantle consists of a cluster of organs including a siphon and 
gill for breathing. Light tactile stimulation of the siphon induces a protective reflex, 
in which the siphon and gill are contracted and withdrawn into the parapodium (see 
Figure 3-3.) 

This reflex has been shown to exhibit several forms of first-order conditioning [30, 
21]. The reflex habituates after repeated tactile stimulation of the siphon, and sensiti- 
zation occurs following an electrical shock to the tail [28]. Simple associative learning 
has also been found [1]. The tail shock is used as the unconditioned stimulus (US), 
and the conditioned stimulus (CS) is tactile stimulation of the siphon. The response 
to associative learning is stronger and lasts longer than with sensitization alone. 

Some types of higher-order conditioning have also been found in Aplysia or its 
close relatives [21, 56]. All of these effects have a short-term form, lasting a few 
hours, and a long-term form which can last for at least several weeks. 

3.1.2 Why Does Aplysia Learn? 

One might wonder just what it is that a sea slug has to learn about, being a simple 
grazing animal with no natural predators. As the marine neurobiologist is a relatively 
recent arrival on the scene, it can hardly be expected to have evolved these abilities 
for the sole purpose of showing off in a petri dish. 

However, even the life of such a gentle beast is fraught with circumstance. Many 
of these creatures live in the littoral, or intertidal, zone of the ocean. In Kandel's 
words, 

Animals living in the littoral zone are often exposed to the air for several 
hours a day and are buffeted about by the tides. These animals must with- 
stand wide and rapid changes in temperature, humidity, salinity, pressure, 
mechanical stimulation, and wind, and survive repeated exposure to air 
and immersion in the sea. [27, page 74] 

It is easy to see why a creature living in such an environment might evolve the 
capacity to learn. If a gentle tactile stimulus, such as might be caused by the begin- 
nings of an undertow, is a reliable predictor that Aplysia is about to be torn from its 
footing and smashed against a rock by an incoming wave, it will clearly benefit by 
recognizing this contingency and tucking its more delicate organs safely out of the 
way. 
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Figure 3-2: Aplysia's body (from [27, page 76].) 
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Figure 3-3: The gill withdrawal reflex (from [27, page 85].) 

Other members of the species live deeper in the sea, in the sublittoral zone, where 
they are rarely exposed to air. Life is a lot calmer in the sublittoral zone. 



3.1.3 How Does Aplysia Learn? 

Aplysia } s nervous system contains about 20,000 central neurons [28]. These are 
grouped into nine ganglia, each of which contains about 2000 neurons. An important 
part of the circuitry for the gill withdrawal reflex resides in its abdominal ganglion. 
The complete circuit is comprised of some fifty or so neurons, including a large number 
of interneurons [19]. The circuit can be reduced to a simplified form containing only 
the primary monosynaptic connections and a few interneurons. The result is shown 
in Figure 3-4. 

The basic reflex is implemented by direct connections from sensory neurons that 
respond to tactile stimulation of the siphon to motor neurons that contract the gill. 

Tactile stimulation at a point on the siphon activates about 8 sensory neurons, 
each of which fires 1 to 2 spikes [28]. 

Sensory neurons from the tail connect indirectly, via facilitatory interneurons, to 
the presynaptic terminals of the siphon sensory neurons. These presynaptic terminals 
are the primary locus for the identified mechanisms underlying classical conditioning 
in Aplysia. 2 



Habituation 

In habituation, the strength of the sensory-to-motor synapse is reduced by repeated 
firing of a sensory neuron. The cause of this change is hypothesized to involve voltage- 



2 Readers who are unfamiliar with neuroscience may wish to refer to Appendix A for background 
that will be helpful in following the material in this section. 
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Figure 3-4: A diagram of the simplified Aplysia neural circuit. Sensory neurons from the 
siphon form direct reflex connections to the motor neurons controlling the gill withdrawal 
reflex. Tail sensory neurons act indirectly on the reflex via facilitator interneurons. The 
interneurons connect to the reflex pathway through synaptic contacts on the presynaptic 
terminals of the sensory outputs. The tail sensory neurons also make direct contacts with 
motor neurons (not shown), (from [21].) 

sensitive calcium channels [28]. 3 These channels are embedded in the cell membrane 
of the presynaptic terminal, and are closed when the cell is at rest. They open 
during action potentials to allow calcium ions (Ca ++ ) to diffuse into the cell from the 
extracellular fluid. 4 The influx of Ca ++ catalyzes neurotransmitter release [27]. 5 

Habituation inactivates these channels, reducing the rate at which Ca ++ diffuses 
into the synapse. This, in turn, reduces the amount of transmitter released in response 
to an action potential. This form of calcium channel inactivation only accounts for 



3 Although these experiments are relatively old, more recent work has concentrated on associative 
and/or long-term learning, so this explanation for the habituation response is still current. 

4 Ion channels can take on a number of distinct conformational states [46]. The probability of 
a state transition is determined by the relative energy and stability of the two states. This Ca ++ 
channel, for example, has three identifiable states: open, closed, and inactivated. The closed state 
has the lowest energy under equilibrium conditions, and is therefore the channel's resting state. The 
state transition from closed to open is most probable when the membrane is depolarized, which 
changes the relative energy of the two states. 

State transitions from the open to the inactive state appear to be common, causing a measur- 
able number of channels to become inactivated over the millisecond time course of a single action 
potential. Transitions from the inactive state back to the resting (closed) state, in contrast, occur 
over a time period of minutes to hours, implying that the inactive state is relatively stable. This 
hypothesis corresponds to the observation that repeated depolarization results in a population shift 
toward the inactive state (that is, given a population of Ca ++ channels, there is an increase in the 
percentage of channels that are inactive after habituation.) 

5 See Appendix C for information about cellular actions of Ca ++ . 
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short-term habituation, as the channels decay back to their normal resting state 
within a few hours. The mechanism for long-term habituation is not known. It may 
act through a process that increases the duration of channel inactivation, or it may 
require some other mechanism. 

Sensitization 

Sensitization apparently acts by a more complex mechanism, using the indirect con- 
nections between the sensitizing stimulus (the tail shock) and the presynaptic ter- 
minals of the siphon sensory neurons [30]. The neurotransmitter released by the 
intervening facilitator interneurons is believed to be serotonin. Serotonin receptors 
in the presynaptic terminal are metabotropic. 6 Unlike ionotropic receptors, which 
act through electrical conductance changes, the serotonin receptor is coupled to a 
G protein which stimulates adenylate cyclase activity. Adenylate cyclase synthesizes 
cyclic AMP (cAMP), a common cellular second messenger which controls a wide 
range of metabolic actions in different cell types [50]. cAMP has been hypothesized 
to act exclusively through a cAMP-dependent protein kinase. Protein kinases, when 
activated, modulate the cell's behavior by phosphorylating a specific set of proteins. 
Phosphorylation changes a protein's activity in a way that is dependent on the par- 
ticular protein and phosphorylation site. 7 

The cAMP-dependent protein kinase in Aplysia apparently affects two types of 
potassium (K + ) channels in the presynaptic terminal. These channels produce the 
end of an action potential by opening to repolarize the cell [22]. Phosphorylation in- 
activates the K + channels, which slows repolarization and thereby prolongs the action 
potential. 

The voltage-sensitive Ca ++ channels remain open as long as the terminal is depo- 
larized, so the Ca ++ influx increases along with the action potential duration. This 
in turn increases transmitter release, resulting in a stronger motor response to the 
sensitized stimulus. 

Again, these mechanisms explain short-term but not long-term sensitization. The 
increase in action potential duration lasts only as long as the increase in intracellular 
cAMP [28]. The cAMP concentration remains elevated for over an hour after a single 
brief pulse of serotonin, however, which is in sharp contrast to the millisecond time 
scale of the electrical effects of ionotropic receptors. 

Associative Learning 

The hypothesized mechanism for associative learning [I] uses the same facilitator 
interneurons that are involved in sensitization. 



6 See Appendix B for background information on neurotransmitter receptors and their actions. 
7 Appendix D contains a brief introduction to protein phosphorylation. 
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Strong stimulation ol the sensory neurons in the tail (the US) activates the gill 
withdrawal motor neurons directly (this is the unconditioned response, or UR), and 
it also activates the facilitator interneurons. This sensitizes the sensory synapses, 
increasing the likelihood that any tactile stimulus will elicit a withdrawal response. 
However, if another sensory neuron (the CS) hres shortly before the facilitator in- 
terneuron, the amplification of the withdrawal reflex for that stimulus is greatly en- 
hanced, with an effect that may last several days. 

The learned association between the US and the CS may occur as a result of 
a second function of Ca ++ . Once it enters a cell, Ca ++ is very quickly bound to 
calmodulin. Like cAMP, the Ca ++ -calmodulin complex is a common cellular second 
messenger. One of the observed actions of Ca ++ -calmodulin is increased activation 
of adenylate cyclase [1, 8]. 

The hypothesized effect of Ca ++ -calmodulin on adenylate cyclase in the Aplysia 
synapse is modulatory. It does not directly activate cAMP production, but it enhances 
the effectiveness of serotonin activation. When Ca ++ -calmodulin is active, therefore, 
a greater amount of cAMP will be produced in response to the same amount of 
serotonin. 

The strength of associative learning depends strongly on the inter-stimulus interval 
(ISI), that is, the time delay between the CS and the US. The associative effect is 
transient, peaking and decaying within two seconds in the aftermath of an action 
potential [1]. Only if the facilitator interneuron hres within this time window is the 
sensitization effect enhanced, resulting in a differential response at that synapse. 

In order for these mechanisms to produce associative learning, the US and the 
CS must each be capable of producing a withdrawal response alone, and there must 
be a facilitator interneuron connecting the US with the output terminal from the 
CS. The need for these pre-existing connections limits the organism's learning ability. 
Only stimulus-response pairings which are already in its behavioral repertoire can be 
learned. There are a huge number of facilitator interneurons in Aplysia [28], which is 
what we would expect if this hypothesis is correct: the more abundant these neural 
connections, the more versatile the animal will be in responding to its environment. 

Another interesting property of these mechanisms is the parsimony of neural con- 
nections: the same connections that are necessary for sensitization, a general arousal 
mechanism, are also used to produce a selectively enhanced response to stimuli that 
consistently precede the US. 

The mechanisms for these three forms of learning (habituation, sensitization, and 
associative learning) are summarized in Figure 3-5. 



Higher Order Learning Phenomena 

Hawkins and Kandel, in response to these findings, speculate on the existence of a 
"cell-biological alphabet for learning" [21]. That is, they hypothesize the existence of 
a small set of primitive cellular mechanisms that can be combined to produce a wide 
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Figure 3-5: Summary of the mechanisms hypothesized to mediate (a) habituation, (b) 
sensitization, and (c) associative learning in Aplysia's gill withdrawal reflex. Each image 
shows the interior of a presynaptic terminal, and diagrams the chemical processes described 
in the text, (from [20, page 76].) 
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range of learning phenomena. They describe how mechanisms already found could, in 
principle, be combined to produce other learning phenomena that have not yet been 
studied. 8 

• Extinction and recovery were hypothesized to work via the same mechanism 
that mediates habituation. 

• Second order conditioning is explained by introducing additional neural connec- 
tions that cause a CS sensory neuron to excite the facilitator interneuron (IN). 
Synapses from the CS to the IN are also facilitated (their behavior is hypothe- 
sized to match a Hebbian learning model, unlike the CS-CR synapses). Thus, 
the ability of the CS to excite the IN is enhanced as it is conditioned. Once 
conditioning is complete, the CS can facilitate other stimuli through the IN in 
the same way the US did originally. 

• Generalization across similar tactile stimuli was explained by the existence of 
overlapping receptive fields for many sensory stimuli, so that a similar stimulus 
would excite a subset of the sensory neurons that were conditioned. 

• Additional properties of the facilitator IN were introduced to explain blocking. 
The IN hres only at the onset of a sustained input, so if a fully conditioned CSi 
hres before the US, it can block the facilitator from firing in response to the US 
neuron. This will in turn prevent facilitation of the new CS2. Because there 
is no time delay between firing of CSi and CS2, the response of the IN to CSi 
does not produce conditioning. If a response in only partly conditioned, the 
facilitator will be only partly blocked, and some conditioning of CS2 will occur. 

• Pre-exposure to the US may cause conditioning to the background environment, 
which in turn would block conditioning to a CS which is presented later. 

Although these learning phenomena have not actually been observed in Aplysia, 
they are found in Limax Maximus, a close relative [16, 56]. 

Mechanisms for Long-Term Learning 

Following the training trials, as the short-term response decays, there is a progressive 
shift toward longer-term forms of learning. This shift occurs gradually over the course 
of several hours. Both the number of times a CS-US stimulus pair is presented and the 
time delay between successive presentations affect the degree to which the animal's 
long-term response changes. 



3 These phenomena are defined in Section 3.1.1. 
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Mechanisms for long-term learning have not yet been extensively explored, though 
it appears to involve both the production of new proteins [12, 65] 9 and new synapse 
formation [17]. 

One apparent mechanism for DNA transcription in long-term sensitization is 
cAMP-dependent phosphorylation of a substrate protein that acts as a DNA tran- 
scription factor [12]. This protein, when activated by cAMP, binds to a particular 
DNA sequence and activates (or, conversely, inhibits) transcription of the associated 
genes. 

What, then, is the function of the proteins encoded by these up- or down-regulated 
genes? One answer is that they may lead to long-term changes in phosphorylation of 
the same set of substrate proteins that is phosphorylated in a short-term manner by 
the presence of cAMP in the presynaptic terminal [65]. In this case, long-term facilita- 
tion would be mediated by long-term spike broadening, via the long-term inactivation 
of K + channels. 

One set of proteins which are down-regulated during long-term sensitization has 
been identified as an NCAM-related class of cell adhesion molecules, which attach 
to the external surface of sensory neurons [29]. NCAMs are important in regulating 
cell growth during development, and may be part of the mechanism for new synapse 
formation. 



3.2 A Simulation of the Aplysia Circuit 

This section describes a simulation of the Aplysia gill withdrawal circuit. Individual 
synapses are the primary site of learning in this network, so the model concentrates 
on the electrical and chemical mechanisms within a single synapse. 

There are at least six distinct learning mechanisms in the circuit, which result in 
short and long term forms of habituation, sensitization, and associative learning, and 
which allow for the possibility of higher-order forms of learning such as differential 
conditioning and blocking. Only the short-term forms of learning are modelled here. 

Some of these mechanisms have previously been modelled elsewhere. Byrne and 
Gingrich [8] model the effects of chemical diffusion and synaptic vesicle transport 
within the synapse, which captures the effect of synaptic depression due to vesicle 
depletion. The model presented by Hawkins [20] demonstrates his hypotheses about 
higher-order learning phenomena, as described in [21]. 

The computational model presented here is a straightforward instantiation of the 
conceptual models used in biology, and a major purpose of the simulation is to test 
the adequacy of current conceptual models for short-term learning. For this rea- 



9 There are two major processes which can be blocked to test for the involvement of protein 
production in a cellular process. The first is DNA transcription, which produces new strands of 
messenger RNA (mRNA). The second is translation of mRNA into amino acid sequences during 
protein synthesis. 
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son, certain aspects of the experimental data are considered more rigorously than in 
other models. Parameter selection is based directly on experimental data, and tested 
against experiments which use different protocols, rather than selected to maximize 
the desired result [20] or left unspecified [8]. The model derived here for the depen- 
dency of transmitter release on Ca ++ influx explains aspects of the experimental data 
that cannot be explained using the simpler models used in [8] and [20]. The process 
underlying associative learning is also modelled in a novel manner that better predicts 
the observed effect of the ISI on the strength of learning. 



3.2.1 Simulation Equations 

Individual action potentials are treated as atomic events which can occur at 10 msec 
intervals. 10 However, state variables, such as concentrations of chemicals in the presy- 
naptic terminal, are updated only when their values are needed to model the cell's 
response to new events. The short-term dynamics of individual learning trials can be 
modelled in this way without the need to step the system through large numbers of 
small time steps during extended periods (e.g. between learning trials) when nothing 
interesting is going on. 

Four state variables are used to model the environment inside the presynaptic 
terminal of a sensory neuron. These variables track the factors hypothesized to me- 
diate conditioning in the synapse. A diagram of the simulated system is shown in 
Figure 3-6. 

When a Ca ++ channel is inactivated during habituation, it ceases to be available 
for influx of calcium during an action potential. [N](t), the concentration of available 
Ca ++ channels, is used to track this effect. 

The sensitization effect involves a cascade of several chemical events, but it is 
rate-limited by the concentration of cAMP [30]. n It is modelled here with a single 
variable, [cAMP](t), representing the cAMP concentration. The dynamics of protein 
kinase activation, protein phosphorylation, and K + channel inactivation are assumed 
to follow cAMP changes closely, and are not modelled separately. 

The factors mediating associative learning are more complex. Again a chemical 
cascade is involved, but here the dynamics of the interaction between Ca ++ , calmod- 
ulin, and adenylate cyclase are modelled to capture the effect of the ISI on associative 
learning. Two state variables are used, [C](t) and [B](t). [C](t) is the concentration 
of free Ca ++ -calmodulin, and [B](t) is the concentration of Ca ++ -calmodulin bound 
to adenylate cyclase. 



10 The duration of an individual action potential is on the order of 3-4 msec, which is much shorter 
than this 10 msec time scale. 

n That is, the change in action potential duration appears to be primarily controlled by an ongoing 
elevation of the cAMP concentration, rather than by some other step in the chemical cascade. 
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Figure 3-6: A diagram showing the state variables used in the simulation model. [N](f) is 
the concentration of active calcium channels; [cAMP](i) is the cAMP concentration; [C](f) is 
the concentration of Ca ++ -calmodulin, and [B](i) is the concentration of Ca ++ -calmodulin 
bonds with adenylate cyclase. [R](f) represents transmitter released into the synaptic cleft. 

Behavior in the Absence of Activity 

When there is no activity in the sensory neuron or the facilitator, state variables 
decay to their equilibrium values according to the following equations: 12 
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where to is the time of the most recent event, [Nl is the total concentration of 

u ' L Jmax 

Ca ++ channels, t^ is the time constant for Ca ++ channel recovery, t c amp is the 
time constant for the cAMP concentration decay, and mi, m 2 , Ci, c 2 , «i, and a 2 are 
determined by the reaction rates between Ca ++ -calmodulin and adenylate cyclase and 



12 These equations are derived from standard rate equations for chemical reactions (see [25] for 
an introduction to this topic). Linear kinetics are assumed for simplicity, except where otherwise 
indicated. 
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the initial conditions at time t (see Section 3.2.3 for a derivation of these parameters.) 

Response to Events 

There are two kinds of events that cause step changes in the internal state of the 
presynaptic terminal. The hrst is an incoming action potential from the sensory 
neuron; the second is an action potential in the facilitator interneuron. 

Sensory Action Potentials The concentration of cAMP is used to determine the 
duration of an action potential in the sensory neuron. 

D(t) = D {1 + K D ■ [cAMP](t)) (3.5) 

where D is the initial duration in the absence of any learning effects, and Kd is the 
magnitude of cAMP's effect on action potential duration. 

The action potential has a primary effect on the rate of Ca ++ influx ([Ic a ](t)), 
which determines both the amount of transmitter released ([R]) and the concentration 
of Ca ++ -calmodulin. It also changes the concentration of inactivated Ca ++ channels, 
which changes the rate of Ca ++ influx for future action potentials. The effect of an 
action potential is modelled as follows: 



[Ica](*) = WN](t) (3.6) 

A[C](t) = [Ic a ](t)-D(t) (3.7) 

A[N](t) = -m(t)(l-K N ) (3.8) 

™ = -mm 

Icao is the Ca ++ influx per Ca ++ channel per unit time, R is the transmitter 
released by a single sensory action potential in the absence of learning, and Kn is the 
percentage of Ca ++ channels still active after an action potential. 

The magnitude of the gill withdrawal response is assumed to be proportional to 
transmitter release. 13 Equation 3.9 is derived in Section 3.2.3. 

Facilitator Action Potentials The other type of event is an action potential in 
the facilitator interneuron. The primary effect of this is a change in the cAMP level, 



13 In biological experiments, this response has been measured in terms of the magnitude of the 
excitatory post-synaptic potential (EPSP) in the motor neuron, the number of motor neuron spikes 
generated by the EPSP, the magnitude of the withdrawal response (that is, how far the gill is 
retracted), or the duration of the withdrawal response. The relationship between number of motor 
neuron spikes and the magnitude of the withdrawal response is approximately linear [27, page 551]. 
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which is modelled as instantaneous. 14 

A[cAMP](t) = [cAMP] (l + K cAMP ■ [B](t)) (3.10) 

[cAMP] is the cAMP released per facilitator action potential in the absence 
of Ca ++ -calmodulin, and K c amp is the percentage increase in cAMP production in 
response to a unit concentration of Ca ++ -calmodulin bonds with adenylate cyclase. 

Parameter Selection 

The parameters in the model correspond to specific physical quantities whose value 
can be determined from the experimental data. 

For simplicity in interpreting the results, the parameters describing the initial 
state of the synapse (in the absence of learning) were normalized, so that all results 
are shown in proportion to the initial state. 

A = D = I Ca0 = Ro = cAMP = 1.0 

The "weak tactile stimuli" used in intact animals are assumed to consist of a single 
action potential in the sensory neuron, in accordance with the response described 
in [28] (one to two spikes per sensory neuron in response to a tactile stimulus). 

Parameters for the calcium channels are set to match the experimental results of 
habituation Experiments 1 and 2 (described in Section 3.2.2 below.) The habituation 
coefficient Kn is set to produce habituation to 30% of the initial response after 15 
trials at an inter-trial interval of 1.5 minutes, and the recovery rate is set to give 75% 
recovery 10 minutes later. 

K N = 0.896 
tjv = 350.0 seconds 

Parameters controlling the sensitization response were set to match the experimen- 
tal results for the calibration test described in sensitization Experiment 1 (described 
in Section 3.2.2.) The value for t c amp was estimated from the graph shown in Fig- 
ure 3-9(a), and Kd was set to produce a maximum response of 450% of the pre-test 
level. 



14 Again, this is a simplification, since the dynamics of the chemical cascade resulting from the G 
protein activation of adenylate cyclase must take some time; the assumption here is that the cAMP 
concentration will have reached its final value before the next event which depends on it. The 
discrepancy between the behavior of this model and sensitization Experiment 2 (described below) 
can be explained if the experimental protocol used in that experiment violates this assumption. 
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K D = 0.0045 
TcAMP = 1000.0 seconds 

There were a number of sensitization and associative learning experiments where 
the strength of the US was not specified. The value used in the simulation was 
chosen to match the strength of the initial response in sensitization Experiment 3. 
The tail shock (and every other noxious stimulus used in experiments other than the 
calibration test) is assumed to consist of 30 facilitator action potentials at 30 Hz. 15 

The parameter K c amp, which controls the strength of the associative response, is 
set using associative learning Experiment 1 (described in Section 3.2.2.) The serotonin 
puff, like other noxious stimuli, was assumed to produce the equivalent of 30 spikes 
over its 1 second duration. 

KcAMP = 13.5 

The parameters which determine the effect of the ISI on learning are discussed in 
Section 3.2.3. 

3.2.2 Results and Discussion 

Habituation and Recovery 

The habituation response in the model was tested against experimental results which 
illustrate the character of the response in Aplysia. These results are: 

1. In [30], light tactile stimulation of the siphon at 1.5 minute intervals caused 
habituation to 30% of the initial response within 10-15 stimuli. 

2. The habituation reflex shows 75% recovery 10 minutes after a single training 
session [27, page 544] (see Figure 3-7(a).) 

3. Stimulating every 30 seconds, the withdrawal reflex is reduced to 20% of the 
maximum response after 10 stimuli. After a 10 minute rest, the reflex recov- 
ers to over 90% of the maximum. The reflex habituates more quickly during 
rehabituation after resting [27, page 544] (see Figure 3-8(a).) 

4. Habituation to 50% of the maximum response occurs with an intertrial interval 
of half an hour [30] (see Figure 3-9(c).) 



15 The simulation assumes that the amount of serotonin released by a single facilitator action 
potential does not change with repeated activity, and that the relationship between the amount 
of serotonin released, the amount of cAMP produced, and the increase in sensory action potential 
duration is linear. 
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Figure 3-7: Results of recovery experiment, (a) Percent recovery of Aplysia's habituation 
reflex, tested at different recovery intervals following a single habituation session, (from [27, 
page 544].) (b) Recovery in the model using same experimental protocol, with a training 
session consisting of 15 trials at 1.5 minute intervals. 

5. Less habituation is observed when stronger tactile stimuli are used [27, page 545] 
(see Figure 3-8(d).) 16 

The recovery time constant Tjv for the habituation tests was calculated using the 
data from Experiment 2, and the habituation coefficient Kff was then set to match 
the habituation data for Experiment 1. These parameters were then tested using 
the experimental protocols for Experiments 3 and 4. 17 Test results are shown in 
Figures 3-7 and 3-8. 



16 The strength of the tactile stimulus is measured in terms of the force of water emitted from a 
water jet, which is used to apply the stimulus. 

17 Experiment 5 was not tested, as it was clear from the start that no mechanism is present in the 
model that can produce this result. 
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Figure 3-8: Results of habituation experiments, (a) Habituation, recovery, and rehabitu- 
ation in Aplysia, using a 30 second intertrial interval and a 10 minute rest after the 10th 
trial (from [27, page 544].). (b) Habituation in the model using the same experimental 
protocol, (c) Habituation in the model with three different inter-trial intervals, (d) Degree 
of habituation found in Aplysia with different stimulus strengths (from [27, page 545].) 
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Discussion The conceptual model used by neuroscientists is unable to explain 
the experimental data on habituation without hypothesizing a more complex set of 
Ca ++ channel states. 

The time course of habituation and recovery in this model is determined by Kn 
and Tjv- These parameters were selected so that the model would match the time 
course of a single habituation experiment well. However, the results of experiments 
using different inter-trial intervals produce different estimates of the parameter values. 
In particular, longer inter-trial intervals result in larger estimates of the recovery time 
constant. Also, the observed time course of recovery follows a roughly exponential 
curve, but the tail of the curve is much longer than would be obtained if the recovery 
rate were constant (as can be seen in Figure 3-7). 

In addition to the complexities in the recovery time, the model is unable to repro- 
duce the experimentally observed enhancement of the habituation response during 
rehabituation after a recovery period. 

These observations may indicate the presence of a multi-stage recovery process, 
where the channel is more susceptible to long-term inactivation in some later recovery 
stage. 

One way to model this would be to use a more complex model of the Ca ++ channels. 
A channel that has two or more inactive states with different recovery time constants, 
for example, could explain both the observed recovery curve and the enhancement of 
channel inactivation during rehabituation. 

An increased susceptibility to long-term inactivation during the late phase of re- 
covery could then be used to explain the presence of habituation with a half-hour 
inter-trial interval. Long-term inactivation would reduce the pool of Ca ++ channels 
participating in rapid recovery, which would shift the entire habituation curve down- 
ward. 

The decrease in habituation observed with strong stimuli also cannot be repro- 
duced with this simple habituation mechanism. In fact, the model predicts greater 
habituation for strong stimuli, not less. The biological effect may be the result of 
unmodelled connections from the sensory neurons onto the facilitator neuron. When 
strong stimuli are used, these connections could produce indirect sensitization of the 
synapse from the sensory to the motor neuron. 

Sensitization 

The following experiments, taken from the Aplysia literature, were used to evaluate 
the model's sensitization response. 

I. In a fully habituated synapse receiving sensory stimulation at 10 second in- 
tervals, a facilitating stimulus was presented at 6 Hz. for 10 seconds. This 
resulted in facilitation to 450% of the habituated response strength, which de- 
cayed over the course of an hour to about 150% of the habituated response. The 
time course closely matched the decay in cAMP over the hrst hour following 
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incubation with serotonin [28] (see Figure 3-9(a).) 

2. A sequence of 13 stimuli was presented at 1.5 minute intervals, followed by 
a strong sensitizing stimulus, 18 followed by another 12 trials [27, page 551]. 
Sensitization following habituation caused a gradual increase in the response 
over the hrst 3 stimuli presented after the sensitizing stimulus before reaching 
a maximum. 

3. Stimulus presentations at half hour intervals following sensitization shows a 
response that is initially almost double the baseline response. Sensitization 
decays over the course of four hours to match the level of habituation that 
would be found if sensitization had not occurred [30]. 

The sensitization parameters Kd and t c amp were determined using Experiment 1, 
and then tested with the protocols used in Experiments 2 and 3. Test results are 
shown in Figure 3-9. 

Discussion The matching time course of decay of sensitization and of the cAMP 
concentration is cited as evidence for a correspondence between the two. However, 
because of the assumed nonlinear dependency between action potential duration and 
strength of the withdrawal response, the time course of the two would not necessarily 
be expected to match closely. 

The observed EPSP shown in Figure 3-9(a) does not return to the baseline level, 
but instead stabilizes at approximately 150% of baseline. This may be evidence for 
the presence of long-term sensitization in this response. 

The gradual increase in the strength of sensitization over the course of the hrst 5 
minutes (see Figure 3-9) does not appear in the model. This observation may indicate 
that the dynamics of the cAMP-dependent cascade do, in fact, produce a significant 
delay. A more accurate model would include two or more stages in the chemical 
cascade from the serotonin receptor to the K + channel. 

As is the case with habituation, there is some disagreement between experiments 
on the duration of the period over which the cAMP concentration is elevated, and 
the corresponding duration of the sensitization response. 

Associative Learning 

The following Aplysia experiments were used to evaluate associative learning in the 
model. 

1. A train of 6 spikes at 10 Hz. (the CS) was immediately followed by a puff of 
serotonin (the US) one second in duration. cAMP levels were measured at four 



18. 



The exact magnitude of the sensitizing stimulus is not specified. 
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Figure 3-9: Results of sensitization experiments, (a) Time course of cAMP concentration 
decay and strength of motor neuron EPSP using 10 second inter-trial intervals (from [28, 
page 51].) (b) Model behavior using same experimental protocol as in (a), (c) Decay 
of sensitization of gill withdrawal reflex following the protocol described in Experiment 3 
(from [30, page 435].) (d) Model behavior using same experimental protocol as in (c). 
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times the concentration found when an unpaired serotonin puff was used [f, 
page 228]. 

2. A weak tactile stimulus in the intact animal was followed immediately by a 
strong unconditioned stimulus, in 15 training trials at a 5-minute inter-trial 
interval, followed by a 30 minute rest period. The strength of the withdrawal 
response was compared to a control group in which the conditioned stimulus 
was presented 2.5 minutes after the unconditioned stimulus. The strength of the 
unpaired response was approximately 1.7 times the pre-test response, whereas 
the associative response was over 4 times the pre-test response (measured in 
terms of duration of gill withdrawal) [1, page 217]. 

3. In Aplysia, with a sequence of 5 training trials at 5 minute intervals, the strength 
of associative learning stabilized at 300% of the initial response [8]. Response 
strength was measured at 5 minute intervals, offset from the training trials by 
2.5 minutes. Results of this experiment in CS+ (paired stimuli at 600 msec ISI), 
CS- (CS follows US by 2 minutes), and sensitization only conditions is shown 
in Figure 3-10(a). 

4. In five training trials at 5 minute intervals, a train of 6 spikes at 10 Hz. was 
administered to a sensory neuron, immediately followed by a strong tail shock. 
Again, a control neuron was stimulated 2.5 minutes after the tail shock. An hour 
after the training trials, the motor neuron EPSP in response to the associated 
sensory stimulus was nearly double its pre-test amplitude, whereas the response 
to the control neuron showed no facilitation [1, page 221]. 

The value of the parameter K c amp was set to match the difference in cAMP 
levels observed in Experiment 1. The resulting value was K c amp = 13.5. The model 
behavior was then tested using the other experiments listed. 

The values for the reflex strength obtained using the protocol described in Ex- 
periment 2 were 1.4 and 1.8, respectively. Results of Experiment 3 are shown in 
Figure 3-10(b). Experiment 4 was not run, because the strength of the associative 
response after half an hour in Experiment 3 was already well below the value observed 
after one hour in Experiment 4. 

Discussion Most of the inaccuracies in the modelling of associative learning could 
be resolved by increasing the time constant for the cAMP concentration decay. The 
experimentally observed value of this parameter is reported as anywhere from about 
twenty minutes to an hour in different experiments. A time constant of an hour or 
more would be necessary to match data on the persistence of associative learning, 
whereas the time constant measured in sensitization experiments is around ten to 
twenty minutes. Part of this difference may be explained by the slow onset of long- 
term learning over the time period in which the short-term response decays. However, 
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Figure 3-10: Results of associative learning experiment, showing 5 training trials followed 
by a sequence of test trials, (a) Strength of gill withdrawal (as percent of initial strength) in 
Aplysia in CS + , CS-, and sensitization only test conditions (from [8, page 66].) (b) Model 
behavior using same experimental protocol. 
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Figure 3-11: Effect of vesicfe depfetion on transmitter reiease. Each curve is for a different 
value of [V](io)j the value of &i[V](io) is kept constant. 

if the presence of a Ca ++ -calmodulin bond increases the duration of adenylate cyclase 
activity as well as its amplitude, this could also explain the increased time constant 
observed with associative learning. 

In Experiment 3, the strength of associative learning appears to stabilize at around 
300% of the baseline reflex strength. The model presented in [8] matches this re- 
sult. They hypothesize that this is because a steady-state concentration of cAMP 
is reached. This would result if the decay in cAMP concentration between trials 
matches the influx during a trial. 

The parameters used in this model for cAMP production and decay, which are 
based directly on experimental data, will saturate at a point where the action po- 
tential duration is approximately three times as long as the baseline duration. This 
would produce saturation of the motor response at 300% only if a first-order relation- 
ship between Ca ++ influx and transmitter release is used. However, as explained in 
Section 3.2.3, experimental evidence suggests that this relationship is higher order. 

As the duration increases, the relationship between it and transmitter release will 
gradually become linear; however, a 200% increase in transmitter release occurs with 
about a 25% increase in action potential duration, which is far from saturating either 
this reaction or the cAMP production process. 

Another obvious candidate for explaining the saturation of the associative response 
is depletion of the available pool of vesicles. In this model, the vesicle pool is assumed 
to be of constant size, so this effect is not modelled. Figure 3-11 shows the effect of 
vesicle depletion on transmitter release, for different initial concentrations of vesicles. 



3.2.3 Derivation of Chemical Interactions 

The details of the chemical reactions involved in classical conditioning in Aplysia have 
a significant effect on the observed macroscopic behavior. Two of these reactions are 
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analyzed below. 

Calcium Influx and Transmitter Release 

The dependency of transmitter release on Ca ++ influx believed to be fourth or- 
der [27]. 19 This makes the synapse highly sensitive to small changes in the Ca ++ 
concentration during an action potential. The resulting dependency between trans- 
mitter release and action potential duration can be as high as fifth-order, as derived 
below. In experimental observations, a 20% increase in the duration of the action 
potential can double the size of the EPSP [I, page 224]. 

Other models [8, 20] assume a first-order dependency of transmitter release on 
action potential duration, which is incompatible with this experimental result. With 
a fifth order dependency, the maximum EPSP amplitude in the experiment shown 
in Figure 3-9(a) requires only a 35% increase in action potential duration to ob- 
tain, whereas with a first-order dependency a more than fourfold increase would be 
required. 

To establish the relationship between action potential duration, Ca ++ influx, and 
transmitter release, we must make certain assumptions about the dynamics of the un- 
derlying chemical reactions. The simplest form for modelling a fourth-order chemical 
reaction is: 

4Ca+++V — R (3.11) 

where V is one vesicle of transmitter, and R is one vesicle of transmitter released into 
the synaptic cleft. The corresponding differential equation is 

ffl^ = MV](T).([I Ca ](T)) 4 (3-12) 

The variable r is used to represent the (continuous) time period over which an 
action potential occurs, as distinct from the discrete time t at which it appears in the 
simulation. 

As vesicles are released into the synaptic cleft, the vesicle concentration [V](r) 
will decrease, but we assume here that it remains relatively constant over the course 
of a single action potential. 20 

Because Ca ++ calmodulin bonds form so quickly, it is assumed that the free Ca ++ 
concentration during an action potential is proportional to the rate of Ca ++ influx, 
and does not build up over time. 21 



19 That is, four Ca ++ ions must bond with each vesicle of transmitter in order to release it into 
the synaptic cleft. 

20 The reduction in [V](t) does, however, play a role in synaptic depression after extended acti- 
vity [8], and may explain part of the mismatch between the behavior of this model and the experi- 
mental results shown in Figure 3-10. 

21 This assumption is also made by both [8] and [20]. 
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and the solution for transmitter release is therefore: 



[R](t) = [ m k[V] ■ ([I Ca ](t)) 4 dr 
Jo 

= *[V]-([Ica](*)) 4 -D(*) (3.14) 

This equation, though it is fourth order with respect to the Ca ++ concentration, 
is first-order with respect to the action potential duration: this result does not give 
us the observed sensitivity to small changes in action potential duration. 

This apparent contradiction can be resolved, however, by modelling the chemical 
reaction in greater detail. The hypothesized mechanism for transmitter release in- 
volves four bonding sites between Ca ++ and each vesicle of transmitter. Transmitter 
will be released when a single vesicle has all four bonding sites filled. This reaction 
has several intermediate forms, with vesicles forming 1, 2, or 3 Ca ++ bonds. If we 
assume that the bonds form independently, then the probability of transmitter release 
from a vesicle V, with four binding sites {V\.. V4}, is 

WM = [vJTkl = iM < 3 - 15 > 

P[R] = P[Vi filkd] A P[V 2 filled] A P[V 3 filled] A P[V 4 filled] 



AM. 



(3.16) 



where [V Q ] is the concentration of open bonding sites, and [Vf] is the concentration 
of filled bonding sites. 

Averaging over the population of vesicles, we get the following equation for trans- 
mitter release: 



din(T) _ /[V f ](r)\ 4 

dr ~ { 4[V] ) (3 - 1?) 

The bonding reaction can be described by the following equation: 
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Ca+++V ^V f (3.19) 

where V Q is one open bonding site, and Vf is one filled site. The differential equations 
for this reaction are 



^[V f ](r) = ^[Ica](*)-[Vo](r)-fe-[V f ](r) (3.20) 

^[Vo](r) = -[V f ](r) (3.21) 

If we assume that all bonds Vf decay between one action potential and the next, 
then [V f ](0) = 0, [V o ](0) = 4[V], and [V ](r) = 4[V] - [V f ](r). In this case, the 
equation for [Vf](r) becomes 

-^-[V f ](r) = 4^[V][Ic a ](t)-(^[Ic a ](t) + fe)[V f ](T) (3.22) 

(IT 

[Vf ](r) = ^i[V][I Ca ](t) _ (fcl[Ica](f)+fc2) 

Kl[ICaJ(^) + h 

and the transmitter released during an action potential is 



™> = rra?y* 



' M \ C ;\ {t \ V / D(f) (l -e-V*MWy dr 

h[l Ca \{t) + k 2 ) Jo y ' 



(3.24) 



Although it is possible to solve this equation exactly, it is more informative to 
look at the two limiting cases, when the time constant (&i[lc a ](0 + k 2 )~ 1 <C D(0 an d 
when (ki[Ica](t) -\- k 2 )~ 1 3> D(t). In the former case, [Vf](r) will saturate very quickly 
relative to the duration of the action potential, and can be approximated as constant. 
Equation 3.24 then becomes: 



[Rim ~ ( h[lc * ]{t) ) f m dr 



h[lcM V D(t) (3.25) 



k 1 [l Ca ]{t) + k 2/ 
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This corresponds to the case where bonds either form or decay very quickly. Note 
that the dependence of transmitter release on [lca](0 i n this case depends on the rela- 
tive speed of the two reactions; if [Ic a ] <C k 2 /ki } then transmitter release is fourth or- 
der relative to [Ic a ] (this corresponds to the solution given in Equation 3.14), whereas 
if [Ica] 3> k 2 /ki } transmitter release is independent of small changes in the calcium 
concentration and depends only on action potential duration. 

In the other limiting case, where both reaction rates are slow, we can do a linear 
approximation to the exponential, and Equation 3.24 simplifies to: 



[ri( " k US+'O f '«<**> +W* 

= ihllc ' ]it))4 (D(t)f (3.26) 

5 

In this case, we have a fifth order relationship between [R](t) and the action 
potential duration, as well as a fourth order relationship with the Ca ++ concentration. 

The experimental data in Aplysia, where a very small percent increase in action 
potential duration produces a large percent increase in the motor neuron EPSP (which 
presumably results from a large percent increase in transmitter released) provides 
evidence supporting the claim that vesicle bonds both form and decay slowly relative 
to the duration of an action potential. This also implies that the proportion of vesicles 
released in a single action potential is small relative to the total number of vesicles 
available. 

One consequence of a fifth order dependence of transmitter release on action po- 
tential duration, assuming that the relationship between cAMP and action potential 
duration is linear (which it may well not be), is that the learning rate 22 remains nearly 
constant even with large changes in synaptic strength. 

Determining the ISI Effect 

In studies of associative learning in Aplysia, the measured effect of the inter-stimulus 
interval on learning reaches a maximum at 0.5 seconds, and decays to about 25% of 
its maximum value after 2 seconds [1]. 

A possible functional reason for the delayed onset of the associative response 
could be to prevent the sensory neuron from learning that it is a predictor of itself. 
This could occur if the sensory neuron stimulates a facilitator interneuron that, in 
turn, makes contacts with the sensory neuron's presynaptic terminals. Facilitator 



22 The learning rate is defined as the percent increase in synaptic strength resulting from a single 
paired stimulus presentation. In the model, this increase is proportional to the total Ca ++ influx 
during a sensory neuron action potential. With a higher-order dependency on action potential 
duration, only a small increase in Ca ++ influx is necessary to potentiate the synapse. 
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interneurons exhibit recurrent inhibition [21], hring only at the onset of a sustained 
input. If interneuron hring ceases within a fraction of a second, the delay in the ISI 
effect will be sufficient to prevent auto-association. 

Hawkins's model [20] uses a 10 second time step, which is sufficient to model the 
learning phenomena at a very rough scale, but prevents accurate modelling of ISI 
effects. In Byrne's model [8], where the associative response is directly proportional 
to the Ca ++ concentration, the maximum response is reached almost immediately 
after an action potential and decays exponentially thereafter. 

In the model presented here, the strength of the associative learning response is 
proportional to the increase in cAMP caused by bonds between Ca ++ -calmodulin 
and adenylate cyclase. Its time course is determined by the dynamics of the interac- 
tions between Ca ++ , calmodulin, and adenylate cyclase. Bonding between Ca ++ and 
calmodulin is assumed to be essentially instantaneous, so that the dynamics of forma- 
tion and dissolution of the Ca ++ -calmodulin-adenylate cyclase bonds and the decay of 
the Ca ++ -calmodulin concentration are the primary factors involved in determining 
the ISI. 

The bonding reaction is 



C + A | B (3.27) 

C -^ inaccessible forms (3.28) 

where A is adenylate cyclase, C is Ca ++ -calmodulin, B is the Ca ++ -calmodulin- 
adenylate cyclase complex, and &i, k 2 and k 3 are the rate constants for the various 
reactions. The second equation represents the removal of Ca ++ -calmodulin from the 
synapse, through a combination of diffusion, uptake into intracellular structures, and 
expulsion of Ca ++ from the cell through an ion pump. 23 

The resulting equations for the dynamics of the reactions are 



dim) 

dt 

d[A](t) 
dt 

d[C](t) 



h[C](t) ■ [A](t] - k 2 [B](t) (3-29) 

-h[C](t) ■ [A](t) + k 2 [B](t) (3.30) 

-ki[C](t) ■ [A](t) + k 2 [B](t) - k 3 [C](t) (3.31) 



dt 
If we assume that there is an excess of adenylate cyclase in the system, so that only 



23 These mechanisms for Ca ++ uptake may have different time constants, but it is assumed here 
that one rate dominates. 
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Figure 3-12: The ISI effect resufting from a simpfe Ca ++ concentration dependency decays 
exponentially as the deiay between the CS and the US increases. Two aiternative modeis 
are shown here, a) ISI effect seen in Byrne's model, which has a simple dependency on 
the Ca ++ concentration, but achieves a slight delay (170 msec.) in the peak response by 
modelling Ca ++ diffusion within the synapse (from [8, page 69].) (b) ISI effect obtained 
with the two-stage reaction used in my model. 
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a small amount is bound by Ca ++ -calmodulin at any given time, we can approximate 
this nonlinear system with the following linear one: 

d[B]{t) = ^[A]-[C](t)-^ 2 [B](t) (3.32) 



dt 

d[C](t) 
dt 



-(^[AHfeMCKO + ^BKt) (3.33) 



The general solution to these equations is given in Equations 3.3 and 3.4, with 
parameter values of: 



Ci = [B](«.)-^[C1W 

(«i - a 2 ) 

C2 = [B]W-.,[C]W 

(a 2 - ai) 

and, dehning a = (&i[A] + k 2 )/2 } 



nil = — « + V ° 2 ~~ ^ 2 ^ 3 (3.36) 



m 2 



-a — \J a 2 — k 2 k 3 (3.37) 



a t = mi + fc ; [A] + fc3 (3.38) 



k 



2 



a 2 = m2 + fc ; [A] + fc3 (3.39) 



k 2 
In the case encountered in the learning trials, [B](t ) = 0, so 



[B](t] = K ( e mi ( f - f °) - e m2(f " fo) ) (3.40) 

K = [C](*„)-^- (3.41) 

a 2 — d\ 



This function reaches its maximum at 

d[B](t max ) 



dt 
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Q _ J£ ( m e m 1 (tmax-t ) _ ™ ^2 (t ma x ~ *0 ) \ 

In \m\\ — In |rr7, 2 1 

tmax ~ ^0 = (3.42) 

m 2 — TTli 

To get a maximum at 0.5 seconds, and a decay to 25% of its maximum at 2 
seconds, we must set ni\ and m 2 according to the following equations: 



[B](*„ + 2) = 0.25 [B](*„ + 0.5) 
e 2mi -e 2m2 = 0.25(e a5mi -e a5m2 ) (3.43) 

and 

In Irai I — In \m 2 \ 

— !-^ !— ^ = 0.5 3.44 

771 2 — ?7^1 

Solving these equations gives us the following parameter values, which are used 
in the simulation: 



nil = 


= -2.93 


m 2 = 


= -1.28 


a,\ - 


= -0.34 


a 2 - 


= 0.52 


'a h = 


= 3.76 


JA1 = 


3.76 + hi 
= 4.22 2 



k 2 

For the value of ki[A] to be positive, k 2 k 3 must fall within a fairly narrow range. 

1.28 < k 2 < 2.93 
1.28 < k 3 < 2.93 

The maximum value for ki[A] that matches the experimental results is 0.34, at 
k 2 = k 3 = 1.94. Using this value for &i[A], and starting with [B](t ) = 0, the 
maximum concentration of bonds (at an inter-stimulus interval of 0.5) is [B] = 
0.06[C](t o ), which means we must have [A] > 0.06[C](t o ), and k x < 5.7[C](t )"\ for 
our first-order approximation to be valid. 

If [A] is not sufficiently high, the system will saturate, and much of the available 
adenylate cyclase will be used up. The effect on the strength of the associative 
response is shown in Figure 3-13. 
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Figure 3-13: Nonlinear distortion of ISI when adenylate cyclase concentration is low. 
The value of &i[A](io) is kept constant. The uppermost curve is without saturation; the 
saturated curves are (in descending order) for [A](io) = .5[C](io) 5 [A](io) = -05[C](£o) 5 an d 
[A](i ) = .01[C](i„). 

The numeric parameters used here are specific to the mechanism which determines 
the strength of the associative learning response. However, the general form of the 
equations is common to many processes that involve chemical second messengers. 
For these equations to apply, two conditions must hold: the activation of the protein 
complex must be dependent on the continued presence of the second messenger, and 
the concentration of the second messenger must decay at a regular rate. 



3.3 Summary 

Traditional computational models of neurons consider only two time scales: the very 
fast time scale of the cell's electrical responses, and the "permanent" changes which 
result from long-term learning. A few exceptions can be found in real-time models 
of reinforcement learning (e.g. [2, 62, 63]), where an exponentially decaying internal 
memory trace is used to model the effect of the ISI on the learning response. The 
mechanisms found in Aplysid's synapses, however, demonstrate the presence of an 
wide range of processes acting at different time scales which have a significant impact 
on the synaptic strength, and therefore on the computational properties, of the reflex 
circuit. 

These mechanisms can be broken down into distinct categories based on their 
time course. Electrical responses and the dynamics of transmitter release are very 
fast, taking place on a millisecond time scale. The chemical processes which determine 
the ISI effect take place on a time scale of a few seconds, while short-term learning 
operates on a time scale of minutes to hours. Long-term learning in Aplysia, which is 
not modelled here, occurs over the course of several hours and lasts for at least several 
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weeks. Processes operating at similar time scales can have complex interactions, 
whereas a large difference in time scales leads to a decoupling of effects. 

Both calcium ions (Ca ++ ) and cyclic AMP are important mediators of Aplysia } s 
learning responses. These chemicals are used throughout the animal kingdom as cellu- 
lar second messengers, controlling many different metabolic processes [50]. Processes 
similar to those modelled here may therefore play a role in many different aspects of 
neural computation. 

The model presented in this chapter reveals the importance of understanding the 
underlying chemical dynamics in Aplysia } s synapses in order to model the macroscopic 
behavior of the circuit for the gill withdrawal reflex. Two specific processes, which 
control the dynamics of transmitter release and of the ISI effect, are examined in 
detail. 

The results of testing the model against data from biological experiments reveal 
enough similarity to support the claim that these mechanisms play an important role 
in conditioning of the reflex, but they also differ in many details. The differences 
may be due to as yet undetected complexity in the underlying chemistry or to the 
contribution of parts of the reflex circuit that are not modelled. 
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Part II 

Topographic Map Formation in 
the Visual System 
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Chapter 4 

A Model of Topographic Map 
Formation 



Topographically ordered projections from one region to another are a prominent fea- 
ture in the brain. The presence of such maps has been known since the 1920's [31], 
when the spatial locations of electrical responses to sensory stimulation in the so- 
matosensory cortex were discovered to lie in an orderly arrangement relative to the 
location of the stimulated body part. These maps have since been found throughout 
the brain. 

The process of map formation in embryonic and early childhood development has 
been most extensively studied in the visual system, in projections from the retina 
to the thalamus and superior colliculus, and in projections from the thalamus to 
the primary visual cortex [66]. These connections form over long distances, and 
initially they are highly disordered. After the initial axonal projections form, they 
sort themselves over time to produce a detailed topographic correspondence between 
the retinal image and the image in the target structure. This map formation process 
makes use of a variety of chemical and electrical mechanisms. 

A simulated model of the effects of chemical and activity- dependent mechanisms 
in topographic map formation is presented in this chapter. These mechanisms work 
together to allow rapid and robust development of detailed maps. This model resolves 
a number of difficulties found in previous modelling attempts. These difficulties in- 
clude controlling the final map orientation, attaining rapid convergence without loss 
of map continuity, and demonstrating a biologically plausible mechanism that pro- 
duces the necessary coactivation pattern between source and target structures during 
activity- dependent map formation. 

Experimental results on the ability to form rough, but not refined, maps in the 
absence of electrical activity, expansion of heavily stimulated areas, and map expan- 
sion and contraction following lesions are matched by this model. Although a highly 
simplified neural model is used, the goal is to explore organizational principles of bi- 
ological development, so some care is taken to remain compatible with experimental 
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Figure 4-1: Axonal projections between brain structures in the early visual system. During 
development, these projections sort themselves to produce topographic mappings between 
the retina, the lateral geniculate nucleus of the thalamus, the superior colliculus (not shown), 
and the primary visual cortex (from [31, page 363].) 

results and with what is known about the underlying biological mechanisms. 

4.1 Introduction 

The motivation for this work comes from recent experimental findings about bio- 
logical mechanisms that give rise to topographic map formation. A variety of such 
mechanisms have been studied in the early visual system, including the maintenance 
of axonal order within the optic tract, positional labelling via chemical markers, and 
activity-dependent synaptic plasticity [66]. 

The presence of chemical markers guiding map formation has been demonstrated 
in a variety of experimental preparations [66, 59]. Although some early researchers 
believed these markers might be sufficient to fully specify the final mapping, in prac- 
tice this does not seem to be true. Evidence for the presence of other mechanisms 
comes partly from studies of expansion and compression of maps following changes 
in input activity or after lesions. Experiments described in [60] point to electrical 
activity in particular as an important contributor to map formation. In this study, 
blocking activity during development prevents the formation of a refined topographic 
map, although rough maps still appear. 

The bulk of experimental evidence, therefore, points to cooperation among a va- 
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riety of mechanisms, physical, chemical, and activity-dependent, for the formation 
and maintenance of topographic maps. Previous computational models, however, 
have attempted to find a single unified map formation process, and fail to exploit the 
advantages of combining multiple mechanisms. 

4.2 Previous Models of Map Formation 

There are four major problems to be overcome by any functional model of biolog- 
ical map formation. These are specifying the orientation of the map, maintaining 
global as well as local continuity, attaining a rapid convergence rate, and specifying 
a biologically plausible implementation. 

Willshaw and von der Malsburg [68] simulated an early activity-based model. It 
solves the problems of map orientation and continuity, but requires the presence of a 
nucleation region in which the initial axonal connections are fully specified. Because 
of the need for a nucleation region, the process converges very slowly. There is no 
biological evidence for such a region. Their marker-based model [67] solves most of 
these problems, but does not match recent experiments establishing the dependence 
of the process on electrical activity. 

Other activity-based models have been based on Kohonen's mapping algorithm 
[36, 37]. This model is mathematically successful, but is not well grounded in biology. 
The input to his algorithm is a vector specification of a single point in the input space, 
rather than a pattern of activity 1 over that space (which corresponds to the input cell 
layer), as is found in biology. The final map orientation is random, and the algorithm 
assumes a specific activity pattern in the target map without considering how that 
pattern could result from a biological process. 

More recent modelling efforts have concentrated on the formation of ocular domi- 
nance and orientation columns, but some have included map formation as well. Ober- 
mayer [51] uses a variant of the Kohonen algorithm which demonstrates map forma- 
tion using a biological input representation. Miller's model [47] displays activity- 
dependent map refinement, but the initial topography is fully specified, and axonal 
arbors narrow over time around a fixed location. 



4.3 The Model 

The model described in this paper is based on the following (biologically motivated) 
description of the developmental process: initial synaptic contacts from the source 
(presynaptic) cell layer are formed over a broad region in the target (postsynaptic) 
layer. Varying concentrations of chemical markers across the surface of the maps 



^n activity pattern is the strength of activation of the neurons, as a function of their location 
in the map. 
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determine the affinity of synapses between particular source and target regions. This 
chemoaffinity provides a rough initial mapping. 

Although some form of chemoaffinity has been found in all biological systems stud- 
ied, the details vary considerably from one system to another. Instead of modelling 
the details of a particular chemical mechanism, therefore, an attempt is made to de- 
termine what constraints all chemical mechanisms must satisfy to ensure a correct and 
detailed final map when combined with an activity-based refinement process. The 
chemical mechanisms are assumed to produce a rough map topography (possibly by 
the mechanism described in [67]), which is used to determine the initial connections 
between source and target layers. Chemical markers do not, in this model, play a role 
in the dynamics of the refinement process. 

Activity- dependent mechanisms then act to refine the initial connections that 
result from chemoaffinity. Synapses are strengthened via a Hebbian mechanism that 
is patterned after long-term potentiation [34]. 2 

The coactivation pattern driving these changes results from correlations in the 
activity of near neighbors in the source map, combined with short-range lateral ac- 
tivation in the target to increase the activity of near neighbors that receive strong 
source inputs. Correlated activity of neighboring neurons in the developing retina, 
prior to the maturation of photoreceptors, has been observed in rat [15, 41], and in 
cat and ferret [42]. The observed activity patterns consist of short correlated bursts 
of activity (lasting a few seconds) separated by relatively long periods of silence. 

Synaptic growth is limited by competition between synapses projecting to the 
same target neuron. Competitive effects cause map expansion under heavy stimula- 
tion, and also cause the breaks in multiple uncorrelated source maps that result in 
ocular dominance column formation. 



4.3.1 The Simulation 

The simulation uses one- dimensional source and target layers to reduce computational 
complexity so that longer-range interactions can be studied. The algorithm does, 
however, generalize to two or more dimensions. Two types of synaptic connections 
are included, as shown in Figure 4-2: modifiable excitatory projections from the source 
layer to the target layer, and unmodihable excitatory lateral connections within the 
target layer. 

Chemical markers are assumed to be present in source and target maps in a linear 
concentration gradient over the surface of the map. The chemical concentration 
is assumed to be detectable by the growing axons with some finite error. These 
assumptions are used to determine the initial connectivity of the map, before the 
activity- dependent process begins. However, the simulation results are valid not only 
for this particular hypothesized mechanism but also for any other mechanism which 



2 See Appendix E for background on long-term potentiation. 
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target layer 
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Figure 4-2: Connections used in the simulation, a) Modifiable projections from source 
(filled circles) to target (open circles) layers. The axonal arbor for a single source neuron is 
shown, b) Fixed-strength lateral connections within the target layer. 
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produces a similar pattern of connections. Initial connections from each source neuron 
cover a broad but finite region in the target, close to (and including) the desired final 
location. 

Lateral connections are localized to a neighborhood around each cell, and their 
strength decreases as the distance between neurons increases. 

Input activity is assumed to occur in spatially and temporally localized clusters 
of neurons. Each input activity cluster occurs at a random location within the source 
layer. For simplicity, all neurons within a cluster are given the same level of activity. 

Each input activity cluster is simulated as a single "trial" of the learning algo- 
rithm. Within a trial, the source is activated, activation is spread through the target 
layer, and connection strengths are updated according to a learning rule which com- 
bines Hebbian learning (in the numerator) with input normalization as a competitive 
mechanism (in the denominator). 

wa(T + l)= i m _f ^ %A > A > A } (4.1) 



» i] (T) + a A 1J (T)o,(T)fl J (T)) 

1=0 
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Wij(T) is the connection strength between source neuron j and target neuron z, a 
is the learning rate, a 3 and o 8 - are the source and target activity, Aij = I if there is a 
synapse between source j and target z, and otherwise, and K is the "competition 
limit", that is, the total synaptic strength over all inputs to a given target neuron. 

The Hebbian component is assumed to arise from long term potentiation (LTP). 
The biological underpinnings of competition are not well understood, though some 
kind of competitive mechanism is necessary to explain experimental results such as 
expansion of a heavily stimulated map region [66]. 

High-level Modelling of LTP 

When using such a highly simplified neural model, some care is necessary to ensure 
that the model maintains some correspondence with its known biological underpin- 
nings. 

Other computational models ignore the transient response to an input stimulus, 
and instead use a hypothetical steady-state frequency response as the target activity. 
However, LTP, the best candidate for a biological form of Hebbian learning, operates 
at a very fast time scale, on the order of 100 msec. 3 The dynamics of the transient 
response is therefore critical, as the system does not have time to settle into steady- 
state within that time window. 4 



3 For example, in the model of LTP described in [7], the strength of the LTP response is assumed 
proportional to the peak Ca ++ concentration; their simulation shows this concentration peaking 
and subsiding within 100-200 msec, after presentation of the input stimulus. 

4 The LTP response assumed by this model is proportional to L Oi(t)dt, for a time window of 
length r. Even if the response does reach steady state near the end of the time window, therefore, 
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Figure 4-3: Typical target activity pattern in response to an input activity cluster. 

In this model, the dynamics of the target response are simulated as follows: 

1. Individual target neurons are modelled as simple thresholded elements. Their 
activation is a weighted sum of their inputs, and if it is above threshold they 
hre. 

2. Each "trial" is divided into time steps. In each time step neurons whose input 
is above threshold hre a single action potential. 

3. Inputs from the source layer arrive at the target neurons at the hrst time step 
of each trial, and initiate hring in neurons over some region. 

4. Firing in the source layer ceases after the hrst time step, but activation continues 
to spread laterally in the target map. At each time step, inputs arrive through 
lateral connections from target neurons that hred in the previous time step. 
Lateral synaptic strengths are set in such a way that the active region in the 
target shrinks over time. 

5. Activation is summed over the duration of the target response to get the activity 
Oi for learning. 

If most of the cells within some region in the target layer are activated, the activity 
will die out in a wave moving inward from the edges of that region. Neurons in its 
center therefore remain active longest. The reasons for assuming this pattern of target 
activity are discussed in Section 4.5. 

Parameters in this simulation were set to give maximum burst lengths of about 
10-20 spikes. At 100 Hz., this response would last no more than 200 msec, which 



the transient response will still dominate. 
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is of the same order of magnitude as the LTP time window. This gives us a rough 
approximation of the actual target activity that would result from this pattern of 
lateral connections. Mathematical details and parameter constraints are given in 
Section 5.1. 



4.4 Results 

Tests were run on one- dimensional source and target layers, with 100 neurons in 
each layer. The ends were wrapped around to reduce edge effects. The location of the 
input activity cluster for each trial was chosen at random in a uniform distribution 
over the input map. Each test ran for 10,000 trials. 

In all cases, the final map orientation is correct, and the map is continuous at a 
global level. Convergence to the correct location is extremely fast, being essentially 
complete within 1000 trials (an average of ten presentations of each input cluster). 

The centroid of the axonal projection from each source neuron, weighted by con- 
nection strength, is used as a measure of its "location" in the target layer; the maxi- 
mum strength connection is generally very close to the centroid. Examples of initial 
and final maps are shown in Figure 4-4. 

The average "spacing error", the variance in distance between centroid locations 
of neighboring source cells, is used to evaluate the accuracy of the local ordering in 
the map. If these locations are equally spaced in the target, the spacing error will 
be zero. At large variances, projections may be "crossed"; that is, their centroid 
locations may be ordered differently in the target than the neurons are in the source. 

Figure 4-5 shows the results of two sets of tests. In one, the source activity cluster 
size is set at a constant, while the initial maximum position error of the projections 
is varied. The accuracy of the resulting map was found to be nearly independent of 
the position error, for values up to ±20 neurons. Follow-on tests (results not shown) 
were done by doubling the size of the target layer (to 200 neurons) and using position 
errors of ±30 and ±40. Again, the final result is near perfect, though it takes slightly 
longer to converge. 

The second set of tests uses a fixed position error, and looks at the effect of the 
source activity cluster size. Larger cluster sizes converged more rapidly, and with the 
smallest cluster size (number of simultaneously active neurons = 2), the final map 
was somewhat disordered. Reasons for this are discussed in Section 4.5.1. 

Another important question in map formation is the shape of the axonal arbors 
(in terms of their connection strengths), and of the receptive fields in the target layer. 
These features are functionally important, and they are easily measured experimen- 
tally. Figure 4-6 shows the initial and final shape of a typical axonal arbor, and the 
strength of a typical receptive held of a target neuron to activity at different loca- 
tions in the source. The receptive held is the combined response to the source and to 
secondary lateral stimulation, so it corresponds to what would be seen in a biological 
experiment. 
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(b) 





(c) 




(d) 



Figure 4-4: Examples of initial and final projections. Filled circles are source neurons, 
open circles are target neurons. Lines are drawn from each source neuron to the centroid 
of its axonal projection, (a-b) initial state, with position errors of ±5 and ±20. (c) typical 
result after 10,000 input stimulus presentations. Results from 15 of 16 tests were very 
similar, (d) worst case result (1 test out of 16), with an initial position error of ±20, cluster 
size = 2 (discussed in text). 
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Figure 4-5: Spacing error, (a) fixed cluster size of 5, with several position errors, (b) fixed 
position error of ±20, with several cluster sizes. 



(a) 



(c) 



(b) 



(d) 



Figure 4-6: Typical arbor and receptive field strengths, (a-b) initial and final connection 
strengths across the target for a given source neuron, (c-d) initial and final receptive field 
strengths of a given target neuron, in response to activity in different locations across the 
source. A source activity cluster size of 5 was used, and activation was allowed to spread 
in the target as it does during the learning trials. 



4.5. THE ROLE OF TARGET ACTIVITY 71 

As is seen in biology, the axonal arbors become more narrow over time, and the 
hnal receptive helds are strongest when input stimulation is at the center of the 
receptive held, and taper off in strength as input stimulation moves away from their 
center [31]. 

4.5 The Role of Target Activity 

In constructing this simulation, we have considerable latitude in modelling the tar- 
get activity pattern, o;(t). In the adult animal, patterns of activity tend to follow 
a characteristic "mexican hat" pattern, with short-range excitatory and mid-range 
inhibitory interactions between inputs. This has led many researchers to hypothe- 
size a pattern of direct lateral connections which are excitatory at close range and 
inhibitory at slightly longer ranges. However, this pattern of activity may be due 
to factors other than lateral synaptic connections, and interactions between inputs 
during development may be quite different than in adults. 

However, the nature of lateral interactions in the target is very important in 
controlling the outcome of map formation. Lateral connections determine the target 
response to a given input stimulus, and the relationship between patterns of activity 
in the source and target maps determines the strength of the synaptic connections 
when map formation reaches an equilibrium state. If input activity is uniformly 
distributed, the dynamics of the system have a fixed point at 

W lheq OC Yl °i,eq(p) (4.2) 

pePj 

where Pj is the set of all input activity patterns in which source node j is active, and 
Oi(p) is the target response to input pattern p. 5 That is, the shape of the arbor for 
a given source node at equilibrium is proportional to the sum of the target responses 
to all input activity patterns in which that node is active. The form of the target 
activity patterns determines the hnal outcome. 

This raises the question of what kind of activity is most useful for map formation. 
The next two sections look at this issue in more detail. 

4.5.1 Noise Reduction 

Map refinement can be thought of as a process of noise reduction. The location of 
the initial arbors provide a rough estimate of the desired target location (i.e. they 
are localized to a neighborhood around it, with some position error); near neighbors 
to a source neuron provide similarly rough estimates of nearby target locations. 

When several source neurons hre in a cluster, their projections can be averaged to 
obtain a more accurate estimate of the correct target location than we could obtain 



3 This result is derived in Section 5.3.3. 
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from any single projection. When this process is repeated with different overlapping 
clusters, the estimate continues to improve. If we simply summed the target activity 
from the source layer, with no lateral connections, this is what we would get. However, 
although the centroid of the arbor converges to the correct location (in a map with 
full connectivity), the distribution of synaptic strength will spread out over time, until 
in the limit the strength is constant over the entire arbor. 6 To compensate for this 
effect, some kind of mechanism is needed to selectively enhance the target response 
in the center of the region activated by an input activity cluster. 

A purely excitatory pattern of lateral connections was chosen because it is ideal 
for producing this type of activity. The only constraints on the source activity are 
that it be sufficient to initiate firing in the target over much of the extent of the 
axonal arbors, and that there be no large gaps in connectivity within an arbor. If 
these constraints are met, and the strength of lateral connections is such that neurons 
in the center of an active region continue to hre, while neurons bordering an active 
region cease firing, this will produce an activity pattern that is strongly enhanced 
in the center of the active region, with decreasing activity enhancement at greater 
distances from its center. 

This kind of activity pattern is very difficult to obtain if a "mexican hat" pattern 
of lateral interactions is used. This pattern of connections tends to produce stable 
activity within a region whose breadth is determined by the extent of lateral excita- 
tion, and inhibit activity outside that region. Periodic activity can also be obtained 
with a mexican hat pattern. This kind of target activity pattern most naturally pro- 
duces a discontinuous map; for example, the patterns of connections found in ocular 
dominance columns could be produced in this way. 

Researchers using this pattern of lateral connections have utilized a variety of 
tricks to ensure a continuous final map. The neighborhood functions used with Ko- 
honen maps [36] are initially very broad, and are manually reduced in size over the 
course of map formation. Synaptic arbors are in this way focussed very slowly into the 
correct region as map formation proceeds, which reduces the difficulty of maintaining 
map continuity. 

Willshaw and von der Malsburg [68] also use a mexican hat pattern of lateral 
connections. In their model, continuity is ensured by the use of a nucleation region. A 
small part of the map is topographically organized as an initial condition, and synaptic 
strengths within this region are much stronger than in the surrounding tissue. This 
ensures that target activity is sufficient to induce learning only at the borders of the 
organized region, which prevents the formation of partial maps that might otherwise 
stabilize and form discontinuous sub-maps. 

Because of the difficulty of maintaining map continuity in the presence of lateral 
inhibition, these results predict that map topography is largely determined before 
the development of an inhibitory surround. It is fairly well established that excita- 



3 See Section 5.3.4 for a derivation of this result. 
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tion predates inhibition in the developing nervous system [18, 40], so this seems a 
reasonable assumption. 

In particular, experimental results on neurotransmitter metabolism in cat visual 
cortex during development [58, 14] were checked for compatibility with this hypoth- 
esis. The observed levels of glutamate metabolism during the stage of development 
in which map formation occurs are much higher than in adult animals, and GABA 
metabolism is still almost nonexistent. As glutamate is the most common excitatory 
transmitter in the cortex, and GABA is a well-known inhibitory transmitter, at least 
in the adult, this result supports the hypothesis that large amounts of excitatory ac- 
tivity are involved in map formation, and rules out one possible mechanism for lateral 
inhibition. However, as it is not actually known whether lateral inhibition in the adult 
is mediated by GABA, nor even whether GABA is inhibitory during development, 
this result is inconclusive. 

Looking at the process as one of noise reduction also explains why the algorithm 
runs into problems with small cluster sizes and large position errors. The smaller the 
cluster size, the less information is available to each step of the noise reduction process, 
because there are fewer "samples" to average over. This means it will converge more 
slowly. However, as the arbors get narrower, due to the enhancement of activity in the 
center of the active region, their position becomes fixed. If this occurs before the noise 
reduction process produces an accurate estimate of the desired location, the map will 
remain somewhat disordered. This problem can be solved by increasing the size of 
the activity clusters. Another prediction of this model, then, is that activity clusters 
must be larger in early stages of map formation than they are later in development. 



4.5.2 Convergence Properties 

Another important effect of the target activity pattern is on the convergence proper- 
ties of the learning algorithm. Erwin, Obermayer, and Schulten [13] have, in the case 
of Kohonen nets on a one- dimensional input space, determined the optimal target 
activity pattern 7 for ensuring a rapid convergence rate and for preventing the for- 
mation of map discontinuities. Although their results have not been demonstrated 
to generalize to this model, they nonetheless provide criteria that are useful for its 
analysis. 

Two requirements are defined that are sufficient to ensure that the final map is 
continuous, and two more to ensure a rapid convergence rate. For a continuous final 
map, the hrst requirement is that the breadth of the activity pattern be greater than 
the uncertainty in position (in their model, this initially corresponds to the entire 
breadth of the map). The second is that it be "convex", a term they define to include 
a slightly larger class of functions than the usual meaning of the term. For rapid 



7 The target activity pattern in this model corresponds to the "neighborhood function" used with 
Kohonen maps. 



74 CHAPTER 4. A MODEL OF TOPOGRAPHIC MAP FORMATION 

convergence, the requirements are that the breadth of the activity pattern be as 
narrow as possible without violating the continuity constraint, and that it be sloped 
toward the center, so that neighboring neurons have maximally differing activity 
levels. 

Since the breadth of the target response in this model is determined directly by 
the breadth of the arbors, which is greater than their positional uncertainty, the hrst 
requirement is easily met. Also, as the arbors narrow down around their correct 
location, the breadth of the response also narrows, which not only meets the breadth 
constraint for rapid convergence, but also tailors itself over the course of learning to 
the current optimum, by narrowing as the position error narrows. 

The shape of the response is a sum of direct activation from the source layer, 
which is (probabilistically) gaussian over the surface of the map, and the response 
due to spreading activation through the lateral connections, which is triangular except 
for some rounding at its peak. The gaussian function is convex within a finite region 
about its center, and the rounded triangular function is convex over its entire nonzero 
region; the target activity is also (probabilistically) convex. The triangular component 
of this activity pattern also gives it a slope that meets the second criterion for rapid 
convergence. 

The activity pattern which arises naturally from intercellular interactions in this 
model thus meets all the requirements given in [13] for obtaining a continuous final 
map with an optimal convergence rate. 

4.6 Extensions to the Model 

4.6.1 Biological Mechanisms for Competition 

The competitive mechanism used in this simulation is a form of input normalization, 
where the total input to a given target neuron is held at a fixed value. Increases in 
synaptic strength due to the Hebbian learning mechanism are offset by decreases in 
the strength of other synapses. Some form of competition is necessary to limit the 
total arbor size, and to explain map expansion and compression in response to lesions 
or changes in the level of input stimulation. 

However, although input normalization is commonly used as a competitive mecha- 
nism in topographic map models, it requires information which is not locally available 
at the synapse, and for this reason it would be very difficult to implement in biol- 
ogy. It is therefore unsatisfactory for a model in which biological plausibility is an 
important consideration. 

Oja [52] showed that vector input normalization can be approximated (in the 
limit where a is small) by an activity- dependent decay, given the assumption that 
there are no lateral connections in the target map. This means that vector input 
normalization can be achieved, if there are no lateral connections, by decreasing the 
synaptic strengths at a rate which depends only on the target activity. 
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Activity-dependent decay suggests a biological mechanism whereby all synapses 
decrease in strength at a rate which is controlled by target depolarization. For ex- 
ample, a voltage-dependent process that affects the turnover rate of postsynaptic 
receptors might have this effect. 

If we follow Oja's derivation, using the form of competition used in this model 
(which normalizes over the summed input strength instead of the vector magnitude), 
we obtain the following rule: 



aN-(T) 
Wij (T + 1) « Wij (T) + a A^T) 0i (T) aj (T) - Wij (T) 0i (T) — ^ (4.3) 

A 

where 

m — 1 

N % (T)=Y,MT)a 3 (T) (4.4) 
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Ni(T) is constant if we assume a constant input cluster size and full connectivity 
between source and target maps, and in this case the decay term becomes a pure 
activity-dependent decay. Its behavior in situations which violate this assumption is 
discussed in more detail in Section 5.2. 



4.6.2 Map Expansion and Compression 

A final set of tests was run to replicate biological experiments showing map expansion 
and compression. There are two ways to get these effects: one is to stimulate a 
localized input area very heavily (or, conversely, deprive an area of input), and the 
other is to introduce a lesion into the source (or target) map, and later observe the 
pattern of connections formed by the remaining tissue [26]. 



Usage-Dependent Changes 

The stimulation-dependent experiments were run simply by changing the probability 
of occurrence of a source stimulus in different regions of the map. Results are shown 
in Figure 4-7. 

In normal cortical maps (where there have been no experimental manipulations), 
the size of the receptive held varies inversely with map magnification [31]. Properties 
of receptive fields within the expanded and compressed regions of the final maps were 
tested to determine how well they match this property of biological maps. 

Receptive held properties were averaged over a set of 20 nodes in the center of the 
compressed region, and 40 nodes in the center of the expanded region. The receptive 
held strength for each source location was determined by stimulating a cluster of 5 
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Trials conpleted: 10000 




(b) 



Figure 4-7: Final results for map expansion and compression after 10,000 stimulus pre- 
sentations, (a) The central half of the input map was stimulated twice as often as the 
surround, (b) The center was stimulated half as often as the surround. 
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Expanded 


Compressed 


Ratio 


Magnification 


1.4 


0.65 


2.2 


Receptive Field Size 


30 


44 


0.68 


Mean Response Strength 


15 


8.5 


1.8 


Proportion Above Mean 


0.47 


0.48 


0.98 



Table 4.1: Map magnification and receptive field properties for expanded and compressed 
maps. 

source neurons centered at that location, 8 and allowing activation to spread laterally 
in the target. Four properties of the final maps were measured: 

f . The average spacing between nodes was used as a measure of map magnification. 

2. The average number of source locations to which a target node responds was 
used as a measure of its receptive held size. 

3. The mean response strength to locations within the receptive held was measured 
separately for each target node, and these values were averaged to determine 
the average mean response strength. 

4. The proportion of source locations for which the response strength was greater 
than the mean response strength for that target was used as a rough measure 
of receptive held shape. 

Results are shown in Table 4.1. This data indicates that the general shape of the 
receptive fields is unchanged by usage-dependent map expansion and compression; 
however, their size and average strength are strongly affected. The strength of the 
receptive fields increases as the map expands, and the receptive held size decreases. 
These properties are similar to biological results in that map magnification and re- 
ceptive held size change in opposite directions. However, the proportional difference 
in receptive held size is considerably smaller than the magnification difference. 

This may, in part, reflect a difference between properties of normal cortical maps 
and properties of experimentally manipulated maps. Magnification of normal cor- 
tical maps varies in proportion to the receptor density in the corresponding source 
region [31]. In the experimental situation, the usage of a source region is modified, 
but there is no corresponding change in receptor density. This could affect the rela- 
tive change in receptive field size as the map expands or compresses. Receptive field 
sizes may also be affected by adaptive changes in the lateral connections, as modelled 
in [61]. 



3 This is the same size that was used to drive map formation. 
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Lesion Studies 

Another change is necessary to run lesion studies. Up to this point, the formation of 
new synapses has not been included in this model. Because of this, it is necessary 
to ensure that the location of the initial arbor overlaps the desired final location. 
However, if some part of the source or target structure is lesioned, the desired final 
location of some arbors is quite far from their initial location, so it is necessary to 
include some form of synapse formation. An extremely simple mechanism for synapse 
formation is used, on the assumption that if it is sufficient, any more realistic model 
is also likely to work. The scheme used is to add new connections, with initially 
random weights, from active source neurons to the immediate neighbors of active 
target neurons. This pattern of synapse formation might result if the source axon 
were to "sprout" new terminals in the vicinity of a synapse that is being strengthened; 
these terminals could then make connections with nearby dendrites of neighboring 
cells. 9 

This scheme, though extremely simplistic, is sufficient to allow map expansion 
and migration into unused tissue after a source lesion, and map compression after a 
target lesion (see Figures 4-8 and 4-9). 

4.7 Discussion and Summary 

Map formation in this simulation is accurate and robust, and provides a good match 
to experimental data, despite the use of an extremely simple neuron model. It may 
be that developmental phenomena such as map formation are relatively insensitive 
to detailed properties of the cell, either because their success is so important to the 
organism, or because these properties are in such flux during development. It is 
likely that modelling of more complex neural behaviors will require the inclusion of 
higher-order effects that appear not to be critical for topographic map formation. 

Chemoaffinity, in this model, is assumed to create a rough initial map that is 
refined by an activity-based mechanism. An LTP-like learning process uses this rough 
map, together with correlations in source activity, to focus the axonal arbors into 
localized regions matching the detailed topography of the source map. The accuracy 
of the initial map is surprisingly unimportant in achieving an accurate final map, as 
long as the axonal projections of neighboring source neurons overlap. Competitive 
mechanisms produce activity- dependent map expansion. 

The pattern of activity in the target cell layer is critical in determining the result 
of the map formation process. The biological process described in this paper results 
in an activity pattern that is ideal for producing rapid convergence to a detailed map. 



9 The model described in [49], for example, uses a similar form of synaptic sprouting as an integral 
part of its learning mechanism. In the optic tectum in the frog, new synapse formation has been 
hypothesized as a primary mechanism for the map development [11]. 
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Figure 4-8: Map expansion following a lesion which removes the right half of the source 
map. Results are shown after (a) 0, (b) 100, (c) 1000, and (d) 10000 trials, (e) Same as (d), 
but with the remaining source neurons spaced apart to full breadth of the map. 
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Figure 4-9: Map compression following a lesion which removes the right half of the target 
map. Axons which would normally connect to the missing portion of the target are initially 
bunched up along the edge of the lesion. Results are shown after (a) and (b) 10000 trials, 
(c) Same as (b), but with the remaining target neurons spaced apart to full breadth of the 
map. 



Chapter 5 

Mathematical Analysis of Map 
Formation 



Chapter 4 presents a simulation of topographic map formation, along with experimen- 
tal results of running that simulation in a variety of test conditions. In this chapter, 
the underlying mathematics are described in detail and analyzed. 

The hrst section presents detailed equations describing the simulation, and gives 
constraints on the parameter values that ensure it will behave according to plan. 

The second section derives a new learning rule which uses an activity-dependent 
decay as a competitive mechanism, and whose behavior is similar to that of input 
normalization given certain simplifying assumptions. Simulation results are presented 
which demonstrate that the new rule works well for map formation. 

The third section addresses the question of the nature and stability of equilibrium 
states for the map formation process. An equation is derived showing the relationship 
between the final weights and the final target activity patterns. For the simple case 
with full connectivity between source and target and no lateral connections in the 
target, it is shown that the system is stable about an equilibrium point where all 
weights have the same value: V 8J - Wij = K/m. 



5.1 Description of Simulation 

The network has a one- dimensional input layer with m nodes, which projects to 
a one- dimensional output layer with n nodes. At trial T, an input activity pat- 
tern [a (T)..a m _i(T)] is presented, and a target response (target activity pattern) 
[oo(T)..o ri _i(r)] is computed by simulating the spreading of activation via lateral 
connections in the target. The strength of the synaptic connection Wij(T) from input 
node j to target node i is then updated by a Hebbian learning rule (in the numerator) 
combined with a competitive mechanism (in the denominator). 
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„,. (rr , ^ _ w ij( T ) + a A-j(T) o,-(T) Qj(T) ,, n 

"W + J- J - -, m _i I - 1 , 1 

-^K(T) + «/l 1] (r)o 1 (T)fl ] (T)) 
j=o 

In this equation, a is the learning rate, Aij(T) = 1 if there is a synapse from j 
to i and otherwise, and K is the maximum synaptic input to a target node. The 
synaptic strengths Wij are strictly excitatory; that is, V 8J Wij > 0. 

Initial conditions for the synaptic connectivity Aj(0) and weights Wij(0) are deter- 
mined using two control parameters, r and t. r is the radius of the axonal arbor, and 
t is the maximum position error. For each input node j, the topographically corre- 
sponding target location is i = round[jn/m\. An error tj for that projection is chosen 
randomly in a uniform distribution over the range [— e..e]. The initial connectivity 
is: 1 



if j + t 3 — r < j < j + t,j + r 
otherwise 



J \ ' 111 AT riQT'HTICa ^' 



Synapses are removed if their strength goes below a minimum value, to prevent 
floating-point underflow errors in the simulation. 

Ai\T + 1) = { l if AtJ ^ = l A Wij ( T ) > 6 (5.3) 

8J 1 otherwise 

In lesion tests, where synapse formation is included, it takes a very simple form. 
The initial weight of a new synapse, Wij(T + 1), is set to a random value. A new 
synapse is formed between a source node j and a target node to which it is not already 
connected when the synapse from j to a neighboring target node is strengthened: 



MT + l] 



1 if (A-,(T) = 1 A Wij (T)>e) 

(5.4) 



\jAw {t _ 1)3 (T)>0 



V Aw {l+1)J (T) >0 
otherwise 



Initial weights Wij(0) are set to random values where A'j(0) = 1? an d 0.0 otherwise. 
The weights are then normalized using the competitive mechanism from the learning 
rule. This mechanism ensures that the summed synaptic strength of all connections 
from the source layer to a given target node is constant, that is, 



-l 



J2 u>ik(T) = K (5.5) 

fc=0 

When K = 1.0, the competitive mechanism is known as "input normalization." 
The input activity aj(T) for a given trial consists of a contiguous activity cluster, 



1 Differences are calculated mod n for wraparound. 
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containing c{T) active nodes. The location of this cluster is chosen randomly. For 
most test runs, c{T) is a constant, and the cluster location is chosen using a uniform 
distribution across the source map. 



«i{T) 



1.0 if node j is active 
0.0 otherwise 



(5.6) 



The target activity Oi(T) for a given trial is determined by the time course of 
target activation within that trial. This time course is simulated by stepping the 
system through the trial with an intra-trial time step t, which is much shorter than 
the inter-trial step T. To get the total target activity for a trial, the activation at time 
t is summed over a time window of length r, beginning at the stimulus presentation 
for that trial. 



Oi(T) 



Ut) 



fl(t) 



t +T 

t=t 



ra — 1 

J2 w t3 (T) A t3 (T) a 3 (T) 

1=0 
n-l 

EU(«-i) 

1=0 

i.o \i<j>j{t)>e 

0.0 otherwise 



t = tn 



t <t <t + T 



(5.7) 



(5.8) 



(5.9) 



In these equations, <f>i(t) is the target activation at time step t within trial T, A 8J 
is the lateral connection strength between target nodes j and z, and 6 is the firing 
threshold for the target nodes. 

The lateral connections A 8J in the target are defined to be a symmetrical triangular 
ramp of radius p. 2 



Xij - < 



A-ll 
0.0 



\ l ~ j\ -1 



if < | i - j | < p 
otherwise 



(5.10) 



The constant A is the summed connection strength of all lateral inputs to a single 
target node. 



n-l 

A = A," = J2 X *3 

1=0 



(5.11) 



^Differences are calculated mod n for wraparound. 
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5.1.1 Parameter Constraints 

There are a large number of parameters in the system, which might at hrst glance 
make it look like a connectionist's nightmare. However, an analysis of these pa- 
rameters gives rise to a simple set of constraints on their values, within which the 
simulation will behave according to plan. The basic parameters are: 

• m (number of source nodes) 

• n (number of target nodes) 

• a (learning rate) 

• K (competition limit) 

• r (length of time window for Hebbian learning) 

• 6 (firing threshold) 

• A (summed lateral connection strength) 

• p (radius of lateral connections) 

Map Configuration 

The map configuration was set at m = n = fOO, with the ends wrapped to reduce 
edge effects, using one- dimensional maps. Using a one- dimensional map makes it 
possible to study effects over a longer distance in the map without the computation 
time becoming prohibitive. 

Learning Parameters 

The competition limit K is the sum of incoming synaptic strengths to a given target 
node. This limit must be set high enough to allow the target to hre in response to 
the source activity, or learning will not occur. 

In the tests which use a uniform input activity cluster size, the competition limit 
is set relative to the cluster size to ensure that initial synaptic strengths are sufficient 
to induce firing in the target. The average initial projection strength is K/2r. For 
an activity cluster of size c, the maximum target activation at time t is therefore 
cK/2r, to initiate target firing we require the following condition: 

cK 

=- > & (5-12) 

Zr 

In these tests, an intermediate K value is used that requires, in the initial state, 
input projections from roughly half the source nodes to initiate firing in a target node. 



5.1. DESCRIPTION OF SIMULATION 85 



4r9 

K = (5.13) 

c 

The target activity function is designed to die out naturally within a small number 
of time steps, and the time window r for Hebbian learning is to be longer than the 
maximum duration of the target activity. 

The Hebb learning rate, a, must be set low enough to ensure convergence, but 
otherwise we want it as large as possible so that it will converge quickly. 

The derivation in Section 5.2 assumes that incremental changes are very small. 
One concrete constraint on the learning rate, required if we are to accurately approx- 
imate the competition term with an activity-dependent decay, is 



--max ^i max 



< 1.0 (5.14) 



K 

or 

a<.- (5.15) 

C 0% max 

For example, if c = 10 and r = 30, assuming a maximum target activity of 50, 

4 -30 -1.0 = 
100 -50 y J 



Parameters Affecting Target Activity 

The firing threshold is normalized to 1.0, and other activity-related parameters are 
set in proportion to it. 

The desired target activity will die out naturally over time, but persist longer 
toward the center of the projection than at its edges. Therefore, if all nodes within 
the lateral activation range are firing, a node should continue to hre, while nodes 
along the borders of an active region should cease firing. In the center of an active 
region, the activation is cf>i = A. 

The activation of a node at the edge of an active region (all nodes on one side 
firing, all nodes on the other quiet), on the other hand, should fall below threshold. 
This activation is cf>i = A/2. 

If we want target activity to die out from the edges, persisting longest in the 
center, the bounds for the lateral connection strengths are therefore 

6<A<26 (5.17) 

An intermediate value for A, 1.5, was used, 
of the summed target activity pattern given different 

The radius of the lateral activation curve, p } is less critical. The primary effect of 
this parameter is to smooth over variations in the input activity — a longer radius 
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increases the ability to recover if individual nodes within an active region do not 
initially reach firing threshold. It also smooths out the summed activity pattern, 
making it more similar to a gaussian, while shorter values of p make the activity 
curve more triangular. 

The value used in the simulation was p = 5. 

Test Conditions 

A number of parameters were varied to create different situations for testing. These 
include 

• r (radius of initial axonal arbors) 

• e (maximum initial position error) 

• c(T) (size of input activity clusters) 

• distribution of locations of input activity clusters 

The hrst set of tests was designed to test the robustness of the learning algorithm. 
The accuracy of the chemical markers and the source activity pattern was varied to 
create different learning conditions. 

The initial radius of the distribution was set at a breadth that would ensure 
overlap with the goal location, to avoid the need to model new synapse formation. 
The corresponding constraint is r > e. 

In a map that wraps around on itself, we also need to ensure that the initial 
position error is not large enough for the axonal arbors to wrap around on themselves, 
as the wraparound is intended to simulate a very large map, not a circular one. To 
prevent this problem, we need the following constraint: 

2(r + e)<n (5.18) 

To test e values between and 20, the projection radius r was set at 30. It would 
be necessary to increase the map size in order to test larger position errors. 

The size and distribution of input activity clusters was tested at constant levels 
between c = 2 and c = 10. The simulation was also tested using input activity cluster 
sizes that varied randomly within that range. The distribution of input activity cluster 
locations was uniform in the initial tests, which should result in a map with uniform 
spacing. Tests of map expansion and contraction used non-uniform distributions. 

5.2 Derivation of Activity-Dependent Decay 

This section shows how input normalization can be approximated by an activity- 
dependent decay term. Although no biological process observed to date is known 
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to produce an activity-dependent decay, it nonetheless allows for a simple biological 
implementation, due to the local availability at the synapse of information about the 
target activity. 

Oja [52] showed that vector input normalization can be approximated (in the 
limit where a is small) by an activity-dependent decay, given the assumption that 
there are no lateral connections in the target map. 

The analysis presented here shows that the decay term for input normalization 
cannot be made completely independent of input activity in the presence of lateral 
connections. However, if we allow the normalization constant K to vary slowly over 
time and across the surface of the target map, we can derive a new learning rule which 
is similar to our old rule but which uses an activity- dependent decay. This new rule is 
simulated to demonstrate that it does produce reasonable competitive behavior, and 
that the summed input strengths, although variable, nonetheless remain close to K. 

Our original learning rule is 

w l3 (T + 1) = m _[ A ^ lA > ,K > A } (5.19) 

77 E K<m « A'^T) o 8 (T) a,(T)) 

IX 1=0 

Because of the input normalization, 



m — 1 

£ W ^T) = K (5.20) 

1=0 



which means we can convert our learning rule to 



Wjj(T ) + a A jj(T) Oi(T) a,j(T) 
a Oi(T) 



WtJ ( T + 1) = '" ' ■ 'v y J Jy J ( 5 - 21 : 

i + ^EA^)^) 

1=0 

All terms in 

nn:(T\ '"iz} 

A l3 (T)a 3 (T) (5.22) 



<*Oi(T) ^ 



1=0 

are finite, so this term can be made arbitrarily small by a suitable choice of a. In 
this case, we can make the following series expansion: 



^.(T)" 1 " 1 



7^77^1 = 1 " HP £ MT) aj(T) + 0[c?\ (5.23) 



1=0 

This transforms our learning rule to 
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Wij (T + l) 



K-(T) + aA t3 (T) o t (T) a 3 (T)) U - ^^ E MT) a 3 (T) + 0[a 2 ] j 

( r p\ ra — 1 

w tJ (T) + aA tJ (T) o t (T) a^T) - ^(Tl^i^A^T) a^T) + 0[a 2 ] 

j=o 

a N(T) 
w l3 (T) + «A 8J (T) ,-(T) a 3 (T) - w tJ (T) 0l (T)^^ (5.24) 



where 



m — 1 

JV,-(T)= E^(^)«i(r) (5-25) 

j=o 

The real appeal of an activity-dependent decay term is that it is dependent only 
on target activity, and therefore has a superior claim to biological plausibility. If 
we assume a constant input cluster size and full connectivity between source and 
target maps, then Ni(T) is constant, and in this case the decay term becomes a pure 
activity- dependent decay. 

In more realistic situations, however, Ni(T) will vary due to variations in the size 
of the input activity cluster and the connectivity between the two maps. 

Up to this point, our derivation is quite similar to the one presented in [52], but this 
is as far as Oja takes us. In Oja's derivation, the decay term was made independent 
from the input activities by assuming that the output is simply the weighted sum of 
the inputs; that is, 



m — 1 



o t (T) = ^ w t3 (T)a 3 (T) (5.26) 

3=0 

which is only the case if there are no lateral connections. Because of the presence of 
lateral connections in this simulation, we cannot make this simplification. 

If we average the effect of the decay term over many time steps, our learning rule 
can be further approximated as: 

a 
w tJ {T + 1) w Wij (T) + aAa(T) o t {T) a 3 {T) - w tJ {T)— (o t {T) N t {T)) (5.27) 

A 

where 

( 0l (T) Ni(T)) = { 0l (T)) (Ni(T)) + Cov [ 0l (T), 7V,-(T)] (5.28) 
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The angle brackets () denote expected value, and Cov[x,j/] is the covariance of x 
and y. Substituting back in to our learning rule, and replacing (oi(T)) with o 8 '(T), 
we obtain 



Wij (T + 1) « Wij (T) + aA %3 (T) o t (T) a 3 (T) - w %3 (T) o t (T) ^ (5.29) 



aNliT) 
K 



where 



(°*( r )) 
with the averages taken over a large time window around T. 

If we average N-(T) over both i and T and replace it by a constant N' = (iV/(T)), 
we obtain a new learning rule with a pure activity-dependent decay. 



aN' 
w tJ {T + 1) = Wij (T) + a Aa(T) o t {T) a 3 {T) - w tJ {T) o t {T)— (5.31) 

A 

The behavior of the new learning rule is approximately equivalent to the old rule 
if the learning rate is small and if N-(T) does not vary over time or across the surface 
of the map. 

However, the summed strength of the inputs to a single target node is no longer 
explicitly normalized to K, so if the actual value of N-(T) differs from our assumed 
value N', the summed input strength will converge to a new value, 

N'-(T) 
K t (T) = K-^l (5.32) 

This gives us a learning rule in which the summed input strength to a target node 
i is maintained at a value which may vary across the surface of the map, or which 
may change slowly over time. 

Two questions are of interest for evaluating this new learning rule. One obvious 
question is how similar it is to input normalization, which is known to work well as a 
competitive mechanism. But other competitive processes which do not normalize the 
inputs to a target node may work equally well. So we must ask, as a separate ques- 
tion, how well this activity-dependent decay term works as a competitive mechanism. 
Figure 5-1 shows the results of a map formation test using an activity-dependent de- 
cay in the place of input normalization. Figure 5-2 shows a graph of Ki(t) across the 
target map at four different times during map formation. These results indicate that 
activity- dependent decay works well for map formation, and although the values of 
Ki do spread out somewhat, they appear to remain within a relatively narrow range 
around a stable mean value. 
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(b) 




(c) 




(d) 



Figure 5-1: Map formation using activity-dependent decay as a competitive mechanism, 
with an initial error of ±20 and an activity cluster size of 5. Results are shown after (a) 0, 
(b) 100, (c) 1000, and (d) 10000 trials. 
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Target node (i) 



Target node (i) 



Target node (i) 



Target node (i) 



(b) 



(d) 



Figure 5-2: Variations in Ki(T) across the target map after (a) 0, (b) 100, (c) 1000, and 
(d) 10000 trials. 
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5.3 Equilibrium Solutions and Stability 

This section derives equations describing the dynamic behavior of the weight vectors 
over the course of a learning simulation. General solutions are given for the equilib- 
rium solutions, and an equation is given which can be used to determine the stability 
of a known equilibrium solution. 

In the case of uniform input activity and full connectivity, it is shown that the 
weights Wij have an equilibrium solution where their values are proportional to the 
sum of the target z's response to all input activity patterns in which the node j is 
active, weighted by the probability of that input activity pattern. This means that 
the process which generates the target activity pattern also determines the shape 
(breadth and distribution of synaptic weights) of the final axonal arbors. 

It is also shown that, for the case where there is uniform input activity, full 
connectivity, and no lateral connections, the map formation process will converge to 
a solution where all weights are equal. 

5.3.1 Equilibrium Solution 

If we assume that incremental changes are very small, and sum a large number of 
time steps t into a single epoch E, we can approximate the system as follows: 



waiE + l) 



w tJ (E) + aJ2P[p] Aij(E) 0i (p, E) aj (p) 
peP 

1 / m_1 \ 

- J2 P[P] ™ij(E) + ao,( P} E) J2 ME) a k (p) 
A P eP V k=o ) 

w tJ (E) + «^P[p] A tJ (E) 0i (p, E) aj (p) 

= — —r (5-33) 

1 + 77 • E PW Oi{p, E) J2 ME) a k (p) 

where P is the set of all possible input patterns, a 3 (p) and o 8 (j>) are the source and 
target activities for pattern p } and P[] denotes "probability of". 

We can define functions to represent the Hebbian and Competition terms in this 
equation as follows: 



Hij(E) = aJ2P[p}ME)o l {p,E)a J {p) (5.34) 

peP 

a m ~ 1 

Ci(E) = -J2 P [p]°^ E )J2ME)a k (p) (5.35) 

A P eP k=o 
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Wij(E) + Hij(E) 
"'■'< £+1 > * l + Ci (E) (5 ' 36) 

Now we can determine the equilibrium states, in terms of Hij(E) and d(E), by 
setting Wij ieq = Wij(E + 1) = Wij(E). This gives us the following: 



^2, eg 



(5.37) 



H lheq = ol Y^ P[p] A lheq o heq (p) a 3 {p) (5.38) 

peP 

a m ~ 1 

Ci, eq = —Y~P\p\o heq (p)Y A lk , eq a k (p) (5.39) 

A P eP k=o 

5.3.2 Stability of Equilibrium Solutions 

To determine stability, we must look at the value of Awij(E) = Wij(E-\-l) — Wij(E) in 
the neighborhood of the equilibrium solutions. Liapunov's method can then be used 
to determine the stability of a known equilibrium solution. To apply this method, 
we linearize the equation for Awij(E) around around a known equilibrium solution, 
then solve for the characteristic roots of the variance matrix. If all real components 
of those roots are negative, then it is possible to construct a Liapunov function for 
the solution, and the state is stable. 
Our variational equation is 



Awij (E) = Wij(E + l)- w tJ (E) 

Wij(E) 



Wij (E) + H^E) 
l + Ci(E) 



H %3 (E) d(E) 

l + Ci(E) l + C,-(£p J 



Wij{E) (5.40) 



The stability of this system cannot be determined in the general case, as the target 
activity, on which both Hij(E) and Ci(E) depend, cannot be found in general except 
by simulation. However, we can determine stability for certain special cases. An 
example is given in Section 5.3.4. 

5.3.3 Equilibrium Solution with Uniform Input Activity 

In this section we solve for the equilibrium solutions in a simplified case, where the 
the input activity cluster size is a constant c, and the input activity is uniformly 
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distributed across the source map. In this case, for a map with its ends wrapped, 
there are m possible input patterns which contain c contiguous active nodes. Each of 
these patterns occurs with probability P[p] = 1/ra. These are the conditions under 
which the hrst set of tests was run. 

Under these conditions, the equations simplify to 



and 



Hij(E) = -J2ME)o l (p,E)a J (p) 



peP 
a 
m 



J2ME)o,( P} E) (5.4i; 

peP 3 



m — 1 

d(E) = -—Y,o^E)Y,A lk (E)a k (p) (5.42) 

K m P eP k=o 

a 
w ij (E) + - ^2 Ai^E) 0i (p, E) 
m TT, 

w tJ (E + l) = P± ^^l (5-43) 

1 + t— E Oi(p, E) J2 ME) a k (p) 
K m P eP k=o 



Hij, eq = — 52 A lheq o heq (p) (5.44) 



P eP 3 

m — 1 



Ci, eq = 77 52 °heq{p) 52 A ^,eq «fc(p) (5.45) 

A m ~p , n 

pEP k=0 



^2, eg 



E • 2_^ Aij : e q Oi^ eq [p) 

pePj 

m — 1 
52 °i,eq(p) Yl Meq a k{p) 

P eP k=o 



(5.46) 



where Pj is the set of all input patterns in which node j is active. 

In addition, we will assume that we have full connectivity between the two layers, 3 
so that Aij = 1, giving us: 

3 A node with a limited synaptic arbor will behave as if it had full connectivity as long as its 
arbor overlaps the same region in the target as other simultaneously active source nodes. 
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Hij(E) = -J2°^ E ) ( 5 - 47 ) 



a m ~ 1 



cm = — j: o.ip, e) j: a k ( P ) 

pE.P fe=0 



a c 



Km ,. p 



"£ 0i (p,E) (5.48) 



a 



w tJ (E + l) = ac '' (5.49) 

A m£ p 



an 



d 



Hij,eq = — Yl °heq(p) (5.50) 



P eP 3 



a c 
K m 



peP 



K J2 °i,eq(p) 

peP 3 

cJ2°i,eq{p) 

peP 



(5.52) 



The weight between any source node j and target node i at equilibrium will be 
proportional to the ratio between the expected activity at i when j is active and the 
average activity at i. 

If we assume a uniform solution, where all arbors have the same shape, then the 
average activity at i is the same for all z, and our weight equation at equilibrium 
simplifies still further, to 

W lheq OC ^ O heq (p) (5.53) 

P ePj 

Since Oi jeq (p) is itself dependent on the u^-j^'s, this equation can be expanded to 
give a solution purely in terms of the weights and the target activity function. 



o eq (p) = F[W eq a(p)] (5.54) 
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W eq oc 



J2 P ePo°e q (p) T 

E P eP m -! °eq{p) T 

J2 pePo F[W eq a(p)V 
E pG P m _ 1 F[W eq a(p)V 



(5.55) 



where o eq (p) is the equilibrium target activity (in vector form) for pattern p } F[x] 
is a function which returns the summed target activity given an initial activation of 
x, and W eq is the weight matrix. 

Given a particular target activation function F, we can calculate W eq for the 
uniform solution, and determine its dependency on the ratio of the competition limit 
K and the cluster size c. 



5.3.4 Equilibrium Solution with No Lateral Connections 

In this section, we will consider the case where we have no lateral connections. 
Now each target node operates independently, so we only need to consider one 
target node, with inputs a(p) = [a (p) a\(p) ■■■ a m _i(p)], weights w(E) = 
[w (E) Wi(E) ■■■ w m -i(E)], and output activity 



m — 1 



o(p, E) = Y, wj{E) a,j{E) = w(E) T a(p) 



(5.56) 



Our learning rule, in this case, simplifies to 



H(E) = a^2P\p]iv(E) T a(p)A(E)a(p) 

peP 



C(E) 



a A(E) IJ2 P \P} %) Apf w(E) 
aA(E)Mw(E) 

^J2 p [p}^( E f^p)MEfs(p) 

A P eP 



(5.57) 



a -> 



K 



7 A(E) T E P b]«(p)«(Pn w(E) 
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- A(E) T M w(E) (5.58) 

A 



w(E) + aA(E)Mw(E) 
l + yA(E) T Mw(E) 

where A(E) is an m by m matrix, with entries 



an 



d 



Solving for the equilibrium states, we get 



a 7*7 
~K 



K A eq M w eq 

K M < 



This system is linear, and we can write its equilibrium solution as 



Al. M w, 

H 

where 



(5.59) 



f A 3 {E) ifi=j 

K(E) = . (5-60) 

otherwise 



M = J2 P W %) a(p) T (5.61) 



# eg = aA e ,M«; e9 (5.62) 



C eq = — A T eq Mw eq (5.63) 



(5.64) 



A eq M u; eg = eq — — w eq (5.65) 



Al q Mw eq = C eq 
K a 

is an eigenvalue of the matrix A eg M, and w eq is the corresponding eigenvector. 



(5.66) 



Stability with Uniform Input Activity 

If we assume a constant input activity cluster size c, full connectivity, and a uniform 
input activity distribution, we can solve for C(E). In this case, 
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M = — Va(p)a(p) T (5.67) 

m ~p 

p£P 



A(E) = n (5.68) 

A(E) = I (5.69) 



where n is a vector of all ones, and 



C(E) = ^A{E) T Mw{E) 
K 



a 



Km c p 



Y, n T a(p) a(p) T w(E) (5.70) 



We can make the following substitutions in this equation: 

n a(p) = c (5-71) 

Y^aipf = cn T (5.72) 

peP 

n T w(E) = K (5.73) 

at which point C(E) can be seen to be a constant: 

<t r 2 

C(E) = (5.74) 

m 

H(E), similarly, can be simplified as follows: 



a 
m 



H(E) = - ( £ m S(pf w(E) 

y P eP J 



where 



—M'w(E) (5.75) 

m 



M' = J2 a(p) a(p) T (5.76) 

peP 



The eigenvalue in this case is just 
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(5.77) 



^eq _ c 

a m 

To determine stability, we want to look at the variation in Aw in the neighborhood 
of the equilibrium states. In this case the system is linear, so we can solve for Aw in 
general, as follows: 



Aw(E) 



H(E) 



C(E) 



l + C(E) l + C(E) 



w(E) 



— M' 

m 


w(E) 


1 + 


a c 2 
m 


a 




m 


(M 



a c 



m 



1 + 



a c 



-w(E) 



m 



2 (M' - c 2 I) w(E) 

1 + — 
m 



(5.78) 



The stability of the equilibrium points can be determined by examining the eigen- 
values of the matrix M' — c 2 I. This matrix is symmetric, therefore all its eigenvalues 
are real. It is stable if all its eigenvalues are less than or equal to zero. 



If we set m = 5 and c 



2, we find that the matrix M is 



M = — 
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1 
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1 
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1 












1 
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1 
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2 



The eigenvalue in this case is c 2 /m = 4/5, and the corresponding eigenvector 
is w = (K/5)n. That is to say, all weights are the same at equilibrium. Now, we 
consider the variational matrix to determine stability. 



M' 





-2 


1 








1 




1 


-2 


1 








2 I = 





1 


-2 


1 













1 


-2 


1 



1 



1 



This matrix is negative semidefmite, so the system is stable, but not asymptoti- 
cally stable. 
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Evolution of the Weight Vector 

An interesting property of map formation in this system can be seen by looking at 
the evolution of the weight vector over time. Combining Equations 5.36, 5.47, and 
5.74 and considering only a single target node, we find 



(£) + -£ w(E) T a(p) 



Wj (E + l) = ^ (5.79) 

1 + 

m 



If we set c = 2, this equation becomes 



a 



Wj (E) + -K-_x(E) + 2w 3 (E) + w j+1 (E)) 
"AE + 1) = m 

1 + — 
m 

2 a a 

1 + — 

= f^w j (E) + ^^(w j . 1 (E) + w j+1 (E)) (5.80) 

1+ — 1+ — 

m m 

What this means is that each of the weights Wj is continuously being averaged 
with its neighbors. This results in a gradual spreading out of the arbor over time, 
until it reaches the equilibrium state where all weights are the same. 



Chapter 6 

Summary and Conclusions 



Most computational models of neurons have assumed that their electrical characteris- 
tics are of paramount importance. However, as the examples in this document show, 
chemical as well as electrical mechanisms play important computational roles in neu- 
rons. Many different mechanisms, both electrical and chemical (and, in some cases, 
mechanical) interact to allow the nervous system to respond to the environment in an 
"intelligent" manner. To adequately model the computational behavior of neurons, 
it is therefore necessary to consider chemical properties as well as electrical ones. 

The processes underlying different aspects of classical conditioning utilize a variety 
of electrical and chemical processes. A functional division can also be made between 
short-term and long-term forms of these learning phenomena. Initial responses to 
conditioning experiments are short-term, but long-term mechanisms gradually come 
into play if learning trials are repeated over an extended time period. 

The Aplysia circuit has been extensively studied, and is unusually well understood. 
The results demonstrate that biological learning is complex and many-faceted, rather 
than working via a single general-purpose mechanism. Even what appears as a single 
mechanism, in a high-level description, is made up of many component processes. 
These processes act at different time scales to modulate a neuron's responses to 
electrical stimulation, working together to regulate the process so that it is stable 
and robust and produces consistent, meaningful behavior. 

Chemical components of learning can be localized within a single synapse, which 
provides neurons with a very fine-grained control over its internal state and, as a 
result, over its electrical responses. These internal "state variables" have a natural 
time period over which they affect the cell's function, and different processes in the 
cell have different temporal properties. Their effects can be controlled by interactions 
between inputs of different types (e.g. utilizing different neurotransmitters), as well 
as by individual inputs. 

The observed high-level learning and behavioral effects may depend on the dy- 
namics of underlying chemical reactions. 

Because they allow the possibility of responding to changes in short-term internal 
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state, the presence of these chemical mechanisms means that a neuron's computa- 
tional properties may resemble those of a finite state machine more than a combina- 
tional logic unit. This is in stark contrast to current computational models of neurons, 
which have little or no internal state outside the permanent state changes modelled 
in long-term learning processes. Some models include very short-term effects such 
as those which are caused by the membrane capacitance, but these generally provide 
little more than a capacity for short-term temporal integration. 

Individual neurons are therefore much more complex than is traditionally as- 
sumed, and have the capacity to implement complex computational functions. These 
computations are specialized for different purposes in different regions of the brain. 

In addition to mediating short-term and long-term changes in a cell's electrical 
behavior, chemical processes play important roles in creating the brain's anatomical 
structure during development. In the developing visual system, activity correlation 
and chemical markers cooperate to create a topographic mapping between different 
structures in the visual input pathway: neither mechanism alone is sufficient. Without 
the initial establishment of a rough topology, activity-dependent mechanisms fall prey 
to a variety of problems and instabilities, and the chemical processes alone cannot 
easily provide the precision necessary for the final map. 

The pre-existing structure of a neural network, such as that specified by chemical 
markers in the visual system, thus directs learning in ways that vastly increase its 
efficiency. The network's structure, conversely, is determined in part by its inputs 
and the learning mechanisms that act within it. The behavior of a neural system 
over time results from a complex interaction between the environment, the network 
structure, and the nature of the learning mechanisms. 



Appendix A 
Modelling of Neurons 



Neurons play a crucial role in the processing of sensory signals and production of 
behavioral responses. It is generally assumed that these functions are mediated by 
their electrical activity, which is described in this appendix. There are many types 
of neurons, and their properties vary considerably, but they all share the same basic 
structure. This appendix is mostly about what neurons have in common, but I also 
discuss some of the simpler variations. See [27] for a more comprehensive discussion 
of this topic. 



A.l Theme 

A. 1.1 Basic Electrical Properties 

The hrst neuron studied in detail was the squid giant axon, which is a popular exper- 
imental preparation due to its large size (it is visible to the naked eye). This meant it 
was possible to do intracellular recordings from this neuron, even when using 1930's 
electrode technology. Our basic understanding of the electrical properties of neurons 
is derived from this example. 

A neuron, like all cells, consists of a lipid membrane which separates the salt water 
in our bodies into intracellular and extracellular compartments. Ions (charged parti- 
cles, such as sodium (Na + ) and potassium (K + )) diffuse freely in water, but the lipid 
acts as an insulator, and prevents ions from flowing between the two compartments. 
Electrically, therefore, the cell membrane behaves as a capacitor, as structurally it 
consists of a thin layer of insulating material sandwiched between two conductive 
layers. 

Ions are able to flow across the cell membrane through ion channels, which are 
proteins that span the membrane and create channels that are often specific to a 
particular type of ion. These channels act as variable resistors, where the channel 
resistance is controlled by such things as the voltage across the cell membrane or the 
presence of substances such as neurotransmitters. In addition to the ion channels, 
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the cell has ion pumps, which actively maintain a concentration gradient of particular 
types of ions across the cell membrane. 

When an ion channel is "open", that is, permeable to a particular ion type, there 
are two forces which control the flux of that ion across the membrane. Diffusion causes 
ions to flow down their concentration gradient, and the voltage across the membrane 
attracts positively charged ions toward the negative side and negatively charged ions 
toward the positive side of the membrane. Because of the ion pumps, which maintain 
the concentration gradient at a near-constant level (the ion fluxes are very small 
compared to their total concentration), each type of ion therefore has an equilibrium 
voltage, called the reversal potential, at which the diffusion force exactly counters 
the electrical force. An open ion channel behaves, electrically, as if its resistance is 
connected to a battery set at the reversal potential for the ions it is permeable to. 

These basic properties of the cell membrane, ion channels, and ion pumps give the 
neuron a great detail of control over its internal electrical state. Electrical commu- 
nication between neurons, on the other hand, is more often mediated by neurotrans- 
mitters, chemicals which are released by one cell to diffuse through the extracellular 
fluid and bind to proteins on another cell's surface. If these receptor proteins open 
or close ion channels in the receiving cell, the extracellular chemical "message" will 
be transduced into an intracellular electrical one. 

There are a number of different neurotransmitters, but each neuron only produces 
a single neurotransmitter. Their effects may be excitatory (that is, they have a 
depolarizing effect on the receiving cell) or inhibitory (hyperpolarizing the receiving 
cell.) 

A. 1.2 Anatomical and Functional Structure 

Understanding how information is processed and transferred within and between 
neurons requires a basic understanding of their anatomical structure. The general 
anatomy of a neuron can be seen in Figure A-l. 

Within a single neuron, there are four basic functional parts: 

• The postsynaptic terminals, most often located in the dendrites and surface of 
the cell body (soma), contain receptors that transduce chemical signals from 
other neurons into electrical currents across the cell membrane. These changes 
can be excitatory (depolarizing) or inhibitory (hyperpolarizing). 1 A cell may 
have hundreds to hundreds of thousands of postsynaptic terminals. 

• The trigger zone, of which there is usually only one, is located in the initial 
segment of the axon near the soma. Action potentials are initiated in the 
trigger zone (that is, it causes the neuron to "hre," producing a brief voltage 
spike) when the electrical potential exceeds a threshold value. 



Tn its resting state, a neuron is polarized, with the intracellular fluid carrying a negative charge 
relative to the extracellular fluid. 
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Figure A-l: What a neuron looks like (from [27, page 97]. 
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• The axon, which acts as an active transmission line, carries the voltage spike of 
the action potential to its presynaptic terminals. 

• The presynaptic terminals, located on branches of the cell's axon, transduce 
the action potential into a chemical signal for intercellular communication. A 
neurotransmitter substance is released into the extracellular fluid, and diffuses 
through this fluid to bind to receptors on another cell, thus completing the 
circuit. Like postsynaptic terminals, a single neuron may have hundreds to 
thousands of presynaptic terminals. 

A synapse is a very important functional unit involving two cells: it consists of a 
presynaptic terminal from one cell and a postsynaptic terminal from the other. These 
terminals are paired, with a narrow gap called the synaptic cleft between them. 

The dendrites are assumed to be the major "input" region of the cell, where signals 
from other cells are received and processed. Electrical signals travel along these long 
thin processes 2 until they reach the cell body, where they are integrated to produce a 
single electrical signal. If the cell is sufficiently depolarized by its inputs, it will hre, 
producing an all-or-nothing output signal which is transmitted down the axon to the 
presynaptic terminals. This is considered to be the source of a cell's "outputs." 

A. 1.3 Mechanisms of Action Potential Generation 

In its resting state, the intracellular fluid carries a negative charge relative to the 
extracellular fluid. The resting potential of a cell is generally around -70 mV. During 
an action potential, this voltage is briefly reversed. 

Two major ion types participate in the generation of action potentials, sodium 
(Na + ) and potassium (K + ). Na + is normally highly concentrated in the extracellular 
fluid, so when Na + channels open, both the concentration gradient and the electrical 
gradient tend to draw ions into the cell. This has a depolarizing action, with a reversal 
potential at around +55 mV. 

K + on the other hand, is highly concentrated inside the cell, so when K + channels 
are open the ions generally flow out of the cell, which has a hyperpolarizing action. 
The reversal potential for K + is around -75 mV. 

There are several different ion channels for each of these two important ions, each 
of which opens and closes under different circumstances. A highly conductive type 
of Na + channel, located along the axon, is voltage-sensitive and transient, opening 
briefly when the potential depolarizes to some threshold value. Na + channel opening 
initiates a positive feedback process whereby Na + influx further depolarizes the cell 
and activates more Na + channels. This process causes the rapid depolarization that 



A process, as defined in Webster's Dictionary, is "a prominent or projecting part of an organism 
or organic structure." 
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characterizes the start of an action potential, and the effect spreads in a fast traveling 
wave down the cell's axon. 

Voltage-activated K + channels then open, with a slight delay, which repolarizes 
the cell as the Na + channels close. This produces the rapid falling edge at the end 
of an action potential. The K + channels remain open somewhat longer, causing an 
"after- hyperpolarization" of the cell which limits the maximum frequency at which it 
can hre. Ion pumps restore the resting concentration of Na + and K + between action 
potentials. 

The voltage spike at each region along the axon brings the next region above 
threshold, so the action potential propagates. However, because the waveform at each 
point results from the local properties of ion channels, local changes in its shape can be 
caused by locally introducing chemicals that affect those channels. For example, in the 
presynaptic terminals of Aplysid's sensory neurons, a type of K + channel is partially 
inactivated during sensitization, which lengthens the action potential's duration. The 
effect of this local manipulation of the action potential waveform is discussed in 
Section 3.1.3. 

When an action potential arrives at a presynaptic terminal, a class of voltage- 
sensitive calcium (Ca ++ ) channel opens. Ca ++ is normally maintained at high con- 
centrations outside the cell, so this causes Ca ++ to flow into the cell. Ca ++ influx 
mediates neurotransmitter release from the presynaptic terminals. 

The neurotransmitter in a presynaptic terminal is contained in vesicles, which are 
tiny "bubbles" of lipid membrane. When Ca ++ is present in the terminal, it binds to 
these vesicles and catalyzes a reaction by which they are attached to release sites and 
emptied into the synaptic gap. Active mechanisms remove free Ca ++ from the ter- 
minal very rapidly after an action potential has passed through, so neurotransmitter 
is released in a brief pulse. 

After their release, neurotransmitter chemicals are removed from the synaptic cleft 
by reuptake mechanisms which gather them back into the presynaptic terminal, or 
by enzymes which metabolize them, breaking them down into substances which are 
inert with respect to intercellular communication. 



A. 2 Variations 
A. 2.1 on the synapse 

A synapse consists of a paired presynaptic and postsynaptic terminal. In addition to 
axodendritic (that is, from the axon to the dendrite) and axosomatic (from the axon to 
the cell body) synapses, described above, some cells have axoaxonal synapses, where 
the postsynaptic terminal is on the axon of a cell, rather than on the dendrites. This 
has the effect of modulating a neurons output directly, instead of through the process 
of signal integration in the soma. One functional implication of this is that it is a 
much finer-grained type of control over a neuron's behavior, as it can affect individual 
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synaptic outputs independently. An example of this can be seen in dopamine release in 
the striatum [55, 10], where local interactions at the presynaptic terminals can cause 
neurotransmitter release that is completely independent of the neuron's electrical 
response at the soma. This release can occur after the cell's axon is cut, at which 
point it no longer even has a soma. 

Invertebrates have long been known to have direct electrical coupling between 
neurons, through a type of synapse called a gap junction. These junctions are com- 
prised of coupled ion channels which span the gap between two cells, forming a direct 
connection between their intracellular compartments. Electrically, gap junctions cre- 
ate a simple resistance between the voltages of the two cells. It was long believed 
that gap junctions did not exist in vertebrates, but evidence has recently been found 
for their presence in mammals during development [53]. 

A. 2. 2 on the inputs 

In sensory neurons, specialized input regions transduce sensory signals into electrical 
potential changes. For example, in the retina of the eye, neurons have specialized 
processes called photoreceptors, which detect impinging photons and emit electrical 
signals in response. Touch sensors convert pressure and vibration into electrical sig- 
nals, and proprioceptive sensors in muscles detect and signal in response to muscle 
tension, stretch and velocity. The firing frequency of these neurons is usually propor- 
tional to either the strength of the input or the time derivative of input strength. 

In simple models of neurons, it is assumed that inputs are simply integrated in 
the soma to produce an electrical depolarization proportional to their sum. However, 
the positioning of postsynaptic terminals along the dendrites may have functional 
effects. For example, an inhibitory synapse may have a shunting effect on excitatory 
signals, which will inhibit an excitatory signal much more effectively if it is located 
along the section of dendrite which lies between the excitatory synapse and the soma. 
This may result in a neuronal response which is proportional to a product of inputs, 
or some combination of sums and products, instead of a simple integration [35]. 

In addition to ionotropic neurotransmitter receptors, which open and close ion 
channels in response to inputs, there is also a class of receptors whose action is 
metabotropic. These receptors initiate chemical processes inside the cell, instead of 
acting through electrical currents. The actions of neurotransmitter receptors are 
discussed in more detail in Appendix B. 



A. 2. 3 on the Outputs 

In addition to electrical currents caused by synaptic inputs, many neurons have in- 
trinsic currents that cause them to hre spontaneously even in the absence of input 
activity [3]. These autoactive neurons show four basic firing patterns: silent (not 
autoactive), regular pulses at some frequency, regular bursts of pulses at some fre- 
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quency, and irregular pulses. This kind of periodic activity in thalamic neurons is 
responsible for the brain waves seen in EEG recordings [24]. 

In the basic neuron model, it is assumed that action potentials propagate globally 
to all presynaptic terminals. However, a model has been proposed [54] where branch 
points in the axon may selectively block action potentials based on the interpulse 
interval, thus performing a more complex output function. 

A. 3 Recapitulation 

This appendix describes the basic properties of neurons. Electrical activity within 
neurons and chemical signalling between neurons are generally assumed to be the 
mediators of a neuron's functional properties. 

Although most neurons have the same basic functional components, the details of 
their structure and behavior vary considerably, and neurons in different brain regions 
can be highly specialized. 

Several aspects of cellular and neural behavior are discussed in more detail in 
other appendices. 
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Appendix B 
Neurotransmitter Receptors 



Neurotransmitter receptors fall into two major classes, ionotropic receptors and meta- 
botropic receptors. There are also receptors such as the NMDA receptor (described 
below) which have both ionotropic and metabotropic effects. 

Many of the examples discussed in this appendix are glutamate receptors, as 
different receptors for glutamate exhibit the full range of receptor types. Glutamate 
is the primary excitatory neurotransmitter in the mammalian cortex, and the synapses 
which exhibit long-term potentiation are glutamatergic. A review can be found in [48]. 



B.l Ionotropic Receptors 

The best-known and most well-understood neurotransmitter receptors are ionotropic. 
Ionotropic receptors are ion channel proteins whose activation is regulated by a neuro- 
transmitter binding site. Ionotropic receptors respond to specific neurotransmitters, 
such as glutamate, by either opening or closing their ion channels. The postsynaptic 
effect of ionotropic receptor activation is a rapid and brief change in the electrical 
currents entering or leaving the cell. 

The glutamatergic AMPA receptor is a highly-conductive Na + channel which 
opens in response to the application of glutamate. It is the primary mediator of 
the depolarizing effects of this excitatory neurotransmitter. Postsynaptic terminals 
in which AMPA receptors are present usually also have other glutamate receptors, 
which mediate other postsynaptic effects. 



B.2 Metabotropic Receptors 

Metabotropic receptors have no direct effect on ion channels or membrane potential. 
Instead, they are coupled to metabolic processes which have indirect effects both on 
ion channel activity and on a variety of other cellular functions. Examples are the 
metabotropic serotonin receptor, described in Chapter 3, the dopamine DI and D2 

Iff 
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receptors, which play an important role in motor control [32, 45], and the recently 
discovered metabotropic glutamate receptor [57], whose activation in the hippocam- 
pus may act to filter incoming signals through selective inhibition or enhancement of 
the cell's electrical response. It has been hypothesized to play a role in regulating 
attention, and in the control of long-term potentiation. 

Many metabotropic receptors are coupled to members of a family of GTP- 
dependent proteins (G proteins). When the receptor is activated, the associated 
G protein complex breaks apart into independent subunits which go on to catalyze 
reactions in the cell. These subunits may in some cases act directly on ion chan- 
nels, producing electrical changes in the cell that persist somewhat longer than those 
of ionotropic receptors. However, they also regulate the cell's activity via second 
messenger systems, activating enzymes such as adenylate cyclase. 

G protein activation of adenylate cyclase stimulates the production of cyclic AMP, 
which controls cellular processes by activating a cyclic AMP- dependent protein ki- 
nase. Protein kinases regulate the activity of a variety of substrate proteins through 
phosphorylation (described in Appendix D.) 

Metabotropic receptors act on a much slower time scale than ionotropic receptors, 
due to the complex cascade of chemical events that mediate their effects. This cascade 
also acts to amplify the incoming signal. For example, a small amount of activated 
G protein can produce a large increase in the cyclic AMP concentration through 
activation of adenylate cyclase. The effects of metabotropic receptors may also persist 
over an extended time period, in part through the catalytic action of protein kinases. 

Activation of metabotropic receptors can have persistent and far-reaching effects, 
ranging from short-lived changes in the sensitivity of other receptors or ion channels 
to large-scale morphological change. 

B.3 The NMD A Receptor 

Some receptor types, such as the glutamatergic NMDA receptor, have both ionotropic 
and metabotropic actions. NMDA receptor activation is necessary for the induction 
of long-term potentiation, a form of synaptic change that requires coactivation of 
presynaptic and postsynaptic cells. 

NMDA receptors contain an ion channel that is primarily permeable to calcium 
ions. Ca ++ influx has both a direct, rapid electrical effect on the cell, and a range of 
indirect metabolic effect 



s. 



If glutamate binds to an NMDA receptor when the cell is near its resting potential, 
the ion channel is blocked by magnesium ions (Mg + ), which, like Ca ++ are present 
in high concentrations outside the cell. Magnesium, however, is unable to fit through 
this channel and enter the cell. When the cell depolarizes, Mg + loses its affinity to 



^^Ca" 1 "" 1 " is extremely important in regulating many aspects of neuronal function. It plays some 
role in all of the processes discussed in these appendices. 
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the channel, and it is freed up for use by Ca ++ ions. Maximum channel activation 
occurs around -20 to -30 mV. Less Ca ++ influx is seen at higher voltages because 
they are nearer its reversal potential, and less influx is seen at lower voltages due to 
the Mg+ block. 

The effect of the NMDA receptor is therefore highly nonlinear. They exhibit al- 
most no activity at low voltage, but as the cell is sufficiently excited by other receptors 
(for example, the AMPA receptor, which is usually present in the same postsynaptic 
terminals), it will "turn on", enhancing the depolarizing effect of strong input sig- 
nals, and initiating longer-term processes in the cell. This dependence on presynaptic 
activity coupled with postsynaptic depolarization, combined with its ability to pro- 
duce long-term changes through the second-messenger effects of Ca ++ ions, give the 
NMDA receptor a unique and important role in neuronal functioning. 
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Appendix C 

The Importance of Calcium in 
Neurons 



Calcium mediates many important intracellular processes. As a charged ion, the 
flux of Ca ++ ions across the cell membrane has a direct electrical effect, acting to 
depolarize the cell. However, unlike Na + and K + ions, whose primary effects appear 
to be electrical, Ca ++ also mediates many important metabolic processes inside the 
cell. Ca ++ ions are necessary to induce transmitter release from vesicles at presynaptic 
terminals. The Ca ++ -calmodulin complex, an important cellular second messenger, 
activates a class of calcium- dependent protein kinases. Ca ++ ions have also received 
attention for their role in the induction of long-term potentiation. Some of these 
processes are discussed in more detail in Appendices D and E. 

C.l Sources of Calcium 

A knowledge of the sources of Ca ++ in the cell is important for understanding the 
control of Ca ++ - dependent processes. 1 

The steady-state intracellular calcium concentration is around 10~ 7 M. Most ion 
channels are inactivated at concentrations above 10~ 6 M, which defines a physiological 
range of concentrations at which biologically meaningful events occur. 

There are two major sources of intracellular calcium: influx through ion channels 
in the cell membrane, and release from internal calcium stores in the endoplasmic 
reticulum or other locations. 

C.l.l Voltage-dependent Calcium Channels 

There are four major types of calcium channels that operate in a volt age- dependent 
manner, opening in response to cell depolarization. The activity of ion channels is 



^^The information in this section is taken from a number of sources, including [4, 39, 64]. 
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defined in terms of their conductance, their location in the cell membrane, and the 
conditions under which they open and close. Two voltage-activated calcium channels, 
the T-type (transient) and L-type (long-lasting) channels, are discussed here. 

T-type calcium channels are low- voltage- activated and transient. These channels 
decay rapidly to an inactive state when cell depolarization is prolonged, and become 
insensitive to further changes in voltage until the cell repolarizes to a level that is 
outside their inactivation range, at which point they return to a resting (closed) state. 
They are believed to be important in producing the spontaneous oscillatory activity 
that underlies periodic bursting in the thalamus. T-type calcium channels may also 
play a role in mossy fiber LTP. 

The L-type calcium channel is one of several high- voltage-activated channels. It, 
too, inactivates under sustained depolarization, but its time constant is extremely 
long, so under most conditions it can be considered a purely volt age- dependent chan- 
nel. The L-type channel is activated in response to large changes in membrane po- 
tential, such as occur during action potentials. It is widely distributed in the central 
nervous system and tends to be concentrated in cell bodies and proximal dendrites, 
especially at the base of the major dendrites. As such, it is believed to play a role in 
action potential generation. 

The L-type channel must be phosphorylated before it can open; this process can 
be mediated by a cyclic AMP-dependent protein kinase. There is evidence to indicate 
that its activation may be actively modulated by endogenous substances. 2 The mech- 
anism for Ca ++ channel modulation is likely to involve neurotransmitters or peptides 
whose receptors are coupled to G proteins. These proteins may affect the channel 
directly or act via an intracellular second messenger such as cyclic AMP. 3 



C.1.2 NMDA Receptors 

The NMDA (for n-methyl-d-aspartate, a drug which activates it) receptor is a neuro- 
transmitter receptor which has a volt age- dependent Ca ++ channel. Unlike the T-type 
and L-type channels, the NMDA receptor requires the presence of glutamate, an exci- 
tatory neurotransmitter, for its activation. This receptor is described in Appendix B. 



C.1.3 Calcium Release from Internal Stores 

Increases in calcium concentration in the cell generally last only a few milliseconds. 
Calcium is rapidly removed from the cell, bound up in protein complexes such as 
Ca ++ -calmodulin, or taken up into internal stores, which may be in mitochondria, in 
the endoplasmic reticulum, or in calciosomes. 



2 Endogenous substances are those which occur naturally in the body. 

3 See Appendices B and D for more information on G proteins, second messengers, and protein 
phosphorylation. 
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Calcium release from internal stores can be induced by caffeine. The function of 
this release may be to induce oscillatory activity in the cell; it is probably not involved 
in transmitter release. 

Calcium release from the endoplasmic reticulum is also induced by a chemical 
called inositol 1,4,5-triphosphate (InsP 3 ). The intracellular concentration of InsP 3 
increases in response to the activation of phospholipase C from cell receptors. This 
is a relatively recently discovered form of cellular second messenger, and its action is 
not well understood. It may be the mechanism by which caffeine acts. 

C.2 Calcium Actions 

The particular changes mediated by Ca ++ influx may vary from cell to cell, and may 
also be modulated by other substances. This section describes a few of the known 
actions of Ca ++ in neural cells. 

Its second-messenger effects and role in long-term potentiation are discussed in 
other appendices, so this section will focus on electrical effects and on the role of 
Ca ++ in transmitter release. 

C.2.1 Electrical Effects 

Many of Ca ++ 's actions in cells are initiated by Ca ++ influx through ion channels 
in the cell membrane. Because it is a positively charged ion, this influx results in a 
voltage change in the cell. There is some controversy over the relative importance of 
electrical and metabolic Ca ++ effects. 

One of the more striking properties of voltage- activated Ca ++ channels is that, 
like the Na + channels which mediate the depolarizing phase of action potentials, their 
opening produces a positive feedback response, which can produce Ca ++ spikes in the 
cell. Ca ++ spikes can occur, for example, in a cell's dendrites in response to input- 
mediated depolarization. They are much slower and smaller in amplitude than Na + 
spikes. 

In addition to calcium spiking due to voltage-activated Ca ++ channels, the voltage- 
sensitivity of NMDA channels can result in a nonlinear dendritic response to synaptic 
inputs. 

Dendrites have traditionally been considered to be passive electrical cables whose 
primary role was the transport of electrical signals into the cell body for integration 
in the trigger zone, but the presence of depolarization-induced dendritic Ca ++ spikes 
indicates that they may play a more active in information processing than has been 
traditionally hypothesized. The properties of active dendrites have been modelled 
and found to have a significant effect on the cell's response to patterns of synaptic 
input with differing spatial distributions [44, 43]. 

Low- voltage-activated Ca ++ channels also play a role in generating periodic oscil- 
lations in spontaneously active cells. The low activation threshold of these channels 
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produces a small "leak" current, which slowly depolarizes the cell. As it depolarizes, 
more Ca ++ channels are activated, which brings the cell to its firing threshold, at 
which point the Na + channels take over and an action potential is generated. These 
Ca ++ channels are transient, but the hyperpolarization of the cell due to K + channel 
opening at the end of an action potential re-activates them, and the process repeats 
at some characteristic frequency. 

Periodic oscillations in cells are important in a class of neural circuits called central 
pattern generators, which control periodic motor responses such as walking, chewing, 
and wing-flapping [3]. 

C.2.2 The Role of Calcium in Neurotransmitter Release 

The presynaptic terminals of neurons are populated with volt age- activated Ca ++ 
channels. When these channels open during an action potential, Ca ++ flows rapidly 
into the terminal. It is removed from the terminal equally rapidly after the channels 
close at the end of an action potential, either by being bound to calmodulin or by a 
variety of other uptake mechanisms. However, during that brief period, the presence 
of free Ca ++ in the terminal causes the release of neurotransmitter into the synaptic 
cleft [33]. 

Ca ++ catalyzes a reaction between vesicles of neurotransmitter and release sites 
on the cell membrane. It requires four Ca ++ ions to catalyze the release of a single 
vesicle of transmitter [27], through a process called exocytosis. Vesicles are small 
free-floating bubbles inside the presynaptic terminals, with neurotransmitter on the 
inside and a lipid membrane on the outside, made of the same substance as the cell 
membrane. In exocytosis, Ca ++ ions increase the affinity between vesicles and release 
sites, where the lipid membrane of the vesicle can merge with the cell membrane, 
dumping its contents outside the cell. 



Appendix D 

Protein Phosphorylation 



Essentially everything that happens inside a cell is mediated by proteins. The struc- 
ture of proteins is what is encoded in the DNA, and the particular proteins present 
in a cell determine its behavior, its morphology, and its function in the body. The 
obvious differences between a skin cell, a muscle cell, and a nerve cell, for example, 
are caused by differences in which particular proteins are present and active in those 
cells. 

The activity of many proteins can be up- or down-regulated by protein phospho- 
rylation, an important mechanism by which extracellular messengers, such as neu- 
rotransmitters and hormones, influence what happens inside a cell. This appendix 
describes the basic mechanisms involved in protein phosphorylation. Much of this 
information was taken from [50]. 



D.l First and Second Messengers 

Chemical first messengers travel in extracellular space and bind to receptors on cell 
surfaces, but they do not themselves carry messages into the cell (though they may 
originate in and be released from cells.) Instead, their message is relayed by second 
messengers, which are produced in response to receptor activation, and which mediate 
the cell's response to the extracellular signal. 

The time course of second messenger effects may be quite long, and they can 
control a wide variety of intracellular processes. Their effects can range from indirect 
opening or closing of ion channels, to up- or down-regulation of the activity of a 
variety of proteins in the cell, to modifications in gene transcription, which may result 
in large-scale changes in cell morphology. Many of these results are accomplished 
through protein phosphorylation. 
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D.2 What is Protein Phosphorylation? 

A class ol proteins, called protein kinases, act as catalysts for protein phosphorylation 
reactions. The basic reaction is the binding of a phosphate molecule to a phospho- 
rylation site on a protein, which results in a conformational change in the protein. 
Many proteins respond to phosphorylation by changes in their activity. The reaction, 
therefore, can be seen as turning the protein "on" or "off", or increasing or decreasing 
the rate at which it acts. 

Each of protein kinase is capable of phosphorylating a specific set of substrate 
proteins, with different affinities for each. A protein kinase may phosphorylate a wide 
range of proteins, or it may be specific to one or a very few. 

The complementary reaction is protein dephosphorylation, which is catalyzed by 
a class of proteins called protein phosphatases. Each protein phosphatase also acts 
on some set of substrate proteins, but their substrate specificity is generally not 
complementary to any particular protein kinase. 



D.3 What does it Do? 

1. A number of ion channels can be activated or inactivated by protein phospho- 
rylation. An example of this is the K + channel discussed in Section 3.1.3. This 
channel is inactivated by phosphorylation, which affects the duration of action 
potentials in the cell, and indirectly affects synaptic strength due to the rela- 
tionship between action potential duration and transmitter release. The L-type 
Ca ++ channel, in contrast, is active only when phosphorylated [4]. 

2. The intermediate-term maintenance of long-term synaptic potentiation in the 
hippocampus (over a period lasting 3-6 hours after induction of LTP) is blocked 
by protein kinase inhibitors, which indicates the importance of protein phos- 
phorylation in mediating this process. LTP discussed in detail in Appendix E. 

3. DNA transcription factors may act as substrate proteins in phosphorylation re- 
actions [12]. These proteins bind to particular DNA sequences and activate or 
inhibit transcription of the associated genes. The role of cyclic AMP in regu- 
lating long-term synaptic potentiation in Aplysia } s sensory neurons, described 
in Section 3.1.3, is an example of this use of protein phosphorylation. 

4. Protein kinases can regulate their own actions through autophosphorylation. 
In the case of cyclic AMP-dependent protein kinase RII, autophosphorylation 
decreases the rate at which the kinase is deactivated, and thus prolongs its 
period of activity. Other possible actions of autophosphorylation are to change 
the affinity of the protein kinase for its substrate proteins, or to maintain its 
activity independent of the continued presence of a second messenger. 
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Phosphorylation of a protein kinase by some other protein kinase could provide 
a mechanism by which the presence of one second messenger might modulate 
the cell's response to another second messenger. 

5. The activity of proteins can also be indirectly regulated by regulating protein 
phosphatases. Several protein kinases have been found that inhibit protein 
phosphatase activity. This will result in prolonged phosphorylation of the sub- 
strate proteins that the phosphatase acts on (which may be distinct from those 
affected directly by the protein kinase.) This provides another mechanism by 
which the action of one second messenger can indirectly control the cell's re- 
sponse to other second messengers. 

6. The activity of many hormone and neurotransmitter receptors and other pro- 
teins that regulate the production of second messengers is also affected by pro- 
tein phosphorylation. This is yet another mechanism by which second messen- 
gers can modulate their own activity or the activity of other second messengers. 

D.4 How is it Controlled? 

There are many substances which regulate protein kinase activity. Among the best- 
known of these are cyclic AMP, cyclic GMP, and the compounds Ca ++ -calmodulin 
and Ca ++ -phosphatidylserine. Each of these is a cellular second messenger, whose 
concentration in the cell is regulated in part by hrst messengers such as neurotrans- 
mitters and hormones. 

Cyclic AMP is produced by adenylate cyclase, and is metabolized (broken down) 
by phosphodiesterase. Cyclic GMP is produced by guanylate cyclase, and is also 
metabolized by phosphodiesterase. Each of these enzymes can itself undergo phos- 
phorylation. Adenylate cyclase and guanylate cyclase are activated by G proteins 
coupled to receptors in the cell membrane. This process is described in Appendix B. 
The concentration of these second messengers may remain elevated for minutes to 
hours after the initial stimulus initiating their production. 

The calcium compounds Ca ++ -calmodulin and Ca ++ -phosphatidylserine are 
formed, along with a number of less well-known second messenger compounds, when- 
ever Ca ++ ions are present in the intracellular fluid. This can come about either by 
the opening of calcium-permeable ion channels in the cell membrane or by release 
of calcium from internal stores. 1 Ca ++ bonds decay rapidly, and free Ca ++ ions are 
actively removed from the cell, so the concentration of these compounds peaks and 
decays rapidly after the influx of Ca ++ ends. 



: See Appendix C for details. 
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Appendix E 
Long-Term Potentiation 



E.l Neural Plasticity and Learning 

Learning in neurons is generally assumed to be mediated by changes in synaptic effi- 
cacy These can result from changes in the signal transmitted by an existing synapse 
or morphological changes in which new synapses are formed or existing synapses grow 
or atrophy. Outside of early development, learning is usually assumed to take place 
primarily in previously existing synapses. 

This learning can, in theory, take place either in the presynaptic terminal, which 
would result in changes in the amount of transmitter released per action potential, 
or in the postsynaptic terminal, which would result in changes in the response for a 
given amount of transmitter [6], or both. 

Presynaptic mechanisms can be very specific, involving a single or small group of 
connections, as individual presynaptic terminals tend to be isolated both electrically 
and chemically. On the other hand, postsynaptic mechanisms can more easily affect 
large populations of synapses [38]. 

E.2 Long-Term Potentiation 

Long-term potentiation (LTP), hrst studied in the hippocampus, is a persistent in- 
crease in the synaptic efficacy of glutamatergic synapses in response to coactivation 
of presynaptic and postsynaptic cells. It has been observed in many regions in the 
brain, and is believed to be an important mechanism for learning. 

Long-term potentiation can be induced either by a burst of activity in an input 
pathway or by input activity coupled with excitation of the target cell from another 
source. The behavior of this kind of LTP is similar to the classic Hebb learning 
rule, where the weight of a synaptic link is enhanced after paired activity in the 
source and target neuron. Because of its Hebbian properties, long-term potentiation 
has generated a great deal of interest in the neuroscience community. As a result, 
our understanding of the mechanisms underlying LTP is changing rapidly. A recent 
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review can be found in [5]. 

E.2.1 Stages in LTP induction and maintenance 

Like the forms of learning seen in Aplysia, long-term potentiation has several stages, 
ranging from short-term potentiation which lasts about an hour, through three dif- 
ferent stages of long-term potentiation, which depend on different types of cellular 
mechanisms. 

Short-term potentiation, lasting 30-60 minutes, is likely to be presynaptic, and is 
independent of protein kinase activity. 

The hrst phase of LTP, in contrast, requires the activity of protein kinases. This 
implies that protein phosphorylation is required to make the transition from short- 
term to long-term forms of learning in this system. In particular, Ca ++ -phospholipid- 
dependent protein kinases have been implicated in the initial phase of LTP, which 
lasts only a couple of hours. 

The second phase of LTP, lasting up to 6 hours, requires protein synthesis. The 
third, and longest-lasting, phase of LTP requires changes in gene transcription. 



E.2.2 Properties of LTP 

LTP is characterized by three basic properties. It is cooperative, meaning that it 
responds to the co-activation of multiple inputs. Low activation levels may induce 
short-term synaptic potentiation, but cooperative activation is needed for long-term 
potentiation. It is associative, in the sense that a weak input may be potentiated by 
coactivation with a strong input. And it is input-specific; that is, potentiation acts 
at the level of individual synapses. 



E.2.3 Induction of LTP 

These properties are believed to be mediated by the response properties of NMDA 
receptors, which require paired presynaptic activity and postsynaptic depolarization 
for activation. Strong evidence has been found for the role of NMDA receptor activa- 
tion in the induction of LTP. However, NMDA alone is not usually sufficient, and, in 
fact, NMDA receptors have been shown to inhibit as well as enhance LTP in different 
situations. It has been hypothesized that coactivation of NMDA and metabotropic 
glutamate receptors may be necessary for LTP induction. 

Both these receptors increase intracellular Ca ++ levels, but they act in different 
ways. NMDA receptor activation causes Ca ++ influx through its ion channel, whereas 
metabolic glutamate receptors may affect Ca ++ levels by increasing Ca ++ release from 
internal stores. This effect is most likely mediated by the second messenger inositol 
1,4,5-triphosphate (InsP 3 ). 
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E.2.4 Maintenance of LTP 

The maintenance of LTP requires persistent changes in synaptic efficacy. The locus 
of these changes is not known, but both presynaptic and postsynaptic modifications 
have been implicated. Different stages of LTP may occur in different locations. 

In the postsynaptic cell, possible mechanisms of action include sensitization of the 
AMPA receptors by phosphorylation in the initial phase, and possibly the synthesis 
of more sensitive subtypes of AMPA receptors in later phases. NMDA receptors may 
also be affected by phosphorylation, reducing the effectiveness of the Mg + block, or 
they may be affeced by agonists such as arachidonic acid or InsP 3 . 

Because the induction of LTP occurs in the postsynaptic terminal, presynaptic 
changes require the release of a retrograde messenger to communicate this activation 
to the presynaptic terminal. The nature of the retrograde messenger is not known, 
although both arachidonic acid and nitrous oxide (NO) have been hypothesized. 

The mechanism for presynaptic changes is also unknown, although changes in 
Ca ++ levels or in the process of transmitter release have been hypothesized. Presy- 
naptic metabotropic glutamate receptors have been hypothesized as a possible medi- 
ator of presynaptic changes. However, which presynaptic mechanisms are most likely 
to be involved depends in part on the (currently unknown) identity of the retrograde 
messenger. 

E.3 Summary 

What is currently known about LTP in the mammalian nervous system implies that 
it acts through metabolic pathways that are very similar to the mechanisms of action 
of classical conditioning in Aplysia. These mechanisms, namely, protein phosphory- 
lation, protein synthesis, and gene transcription, are basic metabolic processes in the 
cell, and the regulation of these processes by neurotransmitters provides a powerful 
mechanism for information processing and the control of neuronal and behavioral 
responses. These responses can be separated into short-term and long-term compo- 
nents, which can be induced or blocked separately and which utilize different meta- 
bolic pathways. The interaction between multiple mechanisms and pathways, and 
the synapse specificity of the responses, provides the neuron with a very fine-grained 
degree of control over its responsiveness to different input patterns. 
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